Division of Research School of Business Administration AN EXAMINATION OF THE NMILOGICAL VALLDITY OF DIRECT PECEUALEASUREMEN Working Paper #580 Michael D. Johnson and David A. Home The University of Michigan FOR DISCUSSION PURPOSES ONLY None of this material is to be quoted or reproduced without the expressed permission of the Division of Research Copyright 1988 University of Michigan School of Business Administration Ann Arbor, Michigan 48109 August 1988

ABSTRICT The paper describes a methodology for the direct measurement of product perceptions. The method uses the spatial or tree-like representations of traditional similarity scaling methods as data collection devices. Consumers record their perception of a product directly within an existing tree or space. A conceptual model of the methodology is developed and tested. The results generally support the model and the ncrrological validity of direct perceptual measurement. * Submitted to the Journal of Marketing Research. Do not quote or reproduce any portion of this manuscript without permission. ** Michael D. Johnson is Associate Professor of Marketing at the University of Michigan's School of Business Administration, Ann Arbor, Michigan, 48109, and David A. Horne is Associate Professor of Marketing at the School of Business Administration, California State University, Long Beach, California, 90840. The authors thank Donald R. Lehmann and Claes Fornell for their crrments on an earlier draft of the paper. The financial support of the Marketing Science Institute and the Executive Education Research Fund of the University of Michigan's School of Business Administration is also gratefully acknowledged.

INrgnCrrIcN Over the past twenty years marketing researchers have applied a variety of similarity scaling techniques to help understand consumer perceptions of product and service alternatives, including multidimensional scaling, hierarchical clustering, and additive tree scaling. Although these techniques offer insight, data requirements often limit their application. The central purpose of this paper is to explore a new, complementary method for collecting perceptual data fraom consumers. The method overcomes sane of the limitations of traditional scaling methods by using product space or tree representations themselves as data collection devices: consumers record their perception of a product's position within an existing representation. Because the method requires consumers to interact directly with a representation, our specific goal is to develop and test a conceptual model of this interaction. We begin by briefly describing traditional similarity scaling methods, their limitations, and a general procedure for direct perceptual measurement. We then describe our conceptual model. Finally, a study that tests the conceptual model is reported. The results generally support the conceptual model and the resulting ncmological validity (Bagozzi 1980; Peter 1981) of direct perceptual measurement. TRADITINAL SIMILARITY MED How do consumers perceive a product relative to its competitors? How do changes in one's marketing mix affect product perceptions? These questions have led marketing researchers to explore and apply a number of techniques for studying perception. One of the more praninent class of techniques is similarity scaling, which includes multidimensional 1

scaling (Cooper 1983; Green 1975; Green and Carmone 1970), hierarchical clustering (Srivastava, Leone and Shocker 1981), additive clustering (Arabie, Carroll, DeSarbo and Wind 1981), and additive tree scaling (Johnson and Fornell 1987). Very generally, the application of these techniques follows a ccnnon procedure. First, consumers produce a series of similarity judgments. These judgments then serve as input to one or more similarity scaling techniques to produce a perceptual representation of the products, typically in the form of a space, a tree, or a hybrid of the two. Multidimensional scaling or MDS, and in particular nonmetric MDS, takes any pair-wise measure of the proximity of a set of stimuli and provides a spatial representation of that proximity (Kruskal 1964; Shepard 1962). The more similar (dissimilar) the original proximity of any two products, the smaller (larger) the relative distance between those products in the resulting space. A sample MDS output for leading candy bars is shown in Figure 1A. A major advantage of MDS is that perceptual data are represented "in a form that is much more accessible to the human eye - namely, as a geometrical model or picture" (Shepard 1972, p. 1). MDS minimizes the stress or badness of fit between the original proximities and the fitted distances based on a prespecified number of dimensions. Solutions of lower dimensionality, such as the two dimensional space in Figure 1A, are the most camron. This is due to the visual appeal and interpretability of low dimensional spaces combined with the typically marginal explanatory power of additional dimensions (Shepard 1972). A second set of related techniques include hierarchical clustering schemes (Johnson 1967), additive clustering (Shepard and Arabie 1979), 2

and additive tree scaling (Sattath and Tversky 1977). All three techniques, like MDS, use a matrix of pair-wise proximities as input. What distinguishes these techniques fran MDS is that they provide treelike representations in which stimuli are represented as external nodes in a tree and distance is equivalent to the length of the paths separating the stimuli. The most familiar of the clustering techniques, hierarchical clustering or HCS (Johnson 1967), imposes certain constraints on the representation. All intracluster distances are constrained to be smaller than all intercluster distances, and all intercluster distances must be equal. The resulting representation is called an ultrametric tree, an example of which is shown in Figure LB. The example uses the same similarity matrix of candy bars used to produce the MDS solution. Algorithms for fitting more general tree structures, called additive trees, were soon developed which loosened the constraints placed on ultrametric trees (Carroll and Chang 1973). These more general additive trees do not constrain the external nodes (stimuli) to be equally distant from any particular root in the tree. Sattath and Tversky (1977) have proposed a heuristic procedure for estimating additive trees called ADDTREE. The ADDTREE solution for the candy bar data is shown in Figure 1C. Although an HCS is simpler in that all of the stimuli are at the same level in the tree, the less constrained ADDTREE often provides a better fit to proximity data (Sattath and Tversky 1977). Unfortunately, the amount of data required by similarity scaling techniques is often foreboding. The number of proximity judgments required to compute a solution quickly becomes an obstacle as the number 3

of products involved increases, or in situations were test-retest or pre- and post-manipulation perceptions are required. With just twelve stimuli, for example, consumers may be asked to provide sixty-six paired comparisons. If judgments are required both before and after sane experimental manipulation, or for more than one set of stimuli, fatigue may result and data quality may become a problem. At least two studies support the effects of fatigue on similarity data. Dong (1983) found more omissions near the end of a similarity judgment task. Meanwhile, Johnson, Lehmann and Horne (1988) demonstrate how judgments simplify through the course of a similarity rating task. Both studies suggest that subject fatigue affects similarity ratings and resulting data quality. While some algorithms can derive solutions frcm simple choice data (cf. DeSarbo and Hoffman 1987; DeSarbo and Rao 1984), this may require a relatively large number of respondents. DIRECT PERCEPTUAL MEU SUREMENT Direct perceptual measurement (DEM) uses the geametric representations and tree structures of existing techniques, not as ends in themselves, but as data collection devices. It is designed for those situations in which a researcher seeks to measure a particular product's perceived position quickly and efficiently, and where more systematic perceptual judgments (e.g., pair-wise similarities) are prohibitive or limiting. Procedure A procedure for collecting DPMs, which represents the generalization of a procedure outlined in Johnson and Horne (1988), is as follows: 4

1. Obtain a suitable external representation (scaling solution) of the product or service alternatives in question. This representation may include the target product of interest. 2. Remove the target product (if included) from the external representation. 3. Present the test consumers with the marketing stimulus of interest, such as a new product concept, a possible product name, or promotional material for an existing, target product. 4. Have the test consumers indicate their perception of the target product' s position directly within the external representation using an intuitive description of that representation. In stage one, any of the existing techniques described earlier, including multidimensional scaling, hierarchical clustering, or additive tree scaling, may be used to provide the external representation. The general goal at this initial stage is to obtain or construct a suitable representation. In many applications, perceptions are considered relatively hcmogeneous. Thus a convenience sample of consumers may suffice for many cacmon products and services. Naturally, however, a DPM can only be as good as the external representation used to collect perceptions. If, for example, product perceptions vary systematically across consumers, separate depictions may be required for more homogeneous subsegments. Overall, the external representation should be in reasonable agreement with the test consumers' own representation of the products. (The sensitivity of DPMs to individual differences, including agreement with the external representation, is examined in the empirical study described below.) Once a suitable external representation is available, the target product, whose perception is to be measured, is removed in stage two. Of course this is unnecessary if the target product was not originally included. In most cases, removing the target product from the stimulus 5

set is unlikely to have any significant effect on the perceived proximity of the remaining products (see Malholtra 1987). Removing the product from a spatial representation is very straightforward. The product's name and position is literally erased or deleted frcm the space. Removing the product frcn a tree structure is only slightly more complicated. The branch containing the product is eliminated frcm the tree and the presentation of the remaining branches is adjusted so that the absence of the missing branch is not obvious (an example of which is described shortly). In stage three, consumers may be presented with a description of an existing product, a description of a new product, the name of a new product, a new name for an existing product, an advertisement for a new or existing product, a price change for a product, a change in company ownership, or other information regarding the target product. In stage four, subjects indicate their perception of the target product directly within the external representation. The methodology is built on the premise that the visual appeal of spatial representations and tree structures allows consumers to quickly and easily indicate their product perceptions. It is important when collecting perceptions to describe the external representations intuitively, at a level the general population of consumers will understand. In a recent application of the methodology (Johnson and Horne 1988), MDS solutions were described simply as pictures or maps. It was explained that products close to each other in the pictures were perceived as similar by consumers while products that were further apart were perceived as more dissimilar by consumers. The consumers placed an X where they thought a product 6

belonged in a map and labeled the X with the product's name. In the present study, both spaces and trees were employed as data collection devices. The spaces were described as pictures or maps while the tree structures were described intuitively as "trees with branches." Consumers were asked to add a branch to a tree to indicate their perception of the target product and then label the branch with the product's name. Illustration Figure 2 illustrates both how the external representations in Figure 1 were modified to collect perceptions and the output of the methodology. In the study, sane subjects were asked to indicate their perception of Snickers candy bar relative to its competitors. The example shows perceptions of Snickers for the same group of subjects within all three representations. The spatial representation (Figure 2A) depicts the individual subject DPMs, Snickers' original position in the space (which was eliminated in order to collect perceptions), and the average DPM (based on the subjects' average X and Y DPM coordinates). The HCS and ADDTREE representations (Figures 2B and 2C) show where each subject added a branch for Snickers' on each tree as well as Snickers' original location. Notice the close proximity between Snickers' original position and average DPM in the space, as well as the large number of subjects who placed a branch close to the original branch locations on the trees. Advantages The DPM methodology offers several advantages over more traditional methods. It allows researchers to very quickly collect multiple product perceptions (see Johnson and Horne 1988 for an 7

example). Unlike traditional scaling techniques, the relative positions of the other products in a representation are held constant. And the method is quite flexible. A wide range of products, services, brands, and categories can be used as stimuli as long as a suitable spatial representation or tree structure is available. The method may also prove useful when tracking a particular product's position over time. Limitations The methodology is by no means all encompassing and was designed with particular data collection problems in mind. It was not, for example, designed for those cases in which several products' positions are simultaneously changing. Perhaps the nost obvious limitation of DPM is that it can only be as good as the external representations used to collect perceptions. Scaling techniques are often criticized because their solutions are not stable across consumers or for the same consumers measured at different points in time (Hauser and Koppelman 1979; Summers and Mackay 1976). If a representation is unstable or provides a poor fit to consumer perceptions, any subsequent direct perceptions are naturally problematic. As mentioned, it may be particularly important that the external representation be an accurate depiction of the test consumers' perceptions. A PERCEP~JAA L 5MEASUREMENP MDEEL The DEM methodology requires consumers to interact directly with an external representation. In this section of the paper we develop a 1. Note that explicitly labeling the dimensions of a spatial representation or the branches of a tree is not the intended purpose of the DPM methodology. The goal of DIM is to capture consumers' overall perception of a product. Explicitly labeling a space or a tree reduces the resultant DPM to a nultiattribute scale rather than an overall perceptual measure. 8

conceptual model that describes this interaction. Testing the model provides an examination of the ncaological validity of direct perceptual measurement. To scientifically clarify what something is, one must specify "the laws in which it occurs" (Cronbach and Meehl 1955, p. 290), or its nomological validity (Bagozzi 1980, Peter 1981). This is a critical first step in developing the methodology. The construct at the heart of the method is perceptual measurement. As a measure of perceived similarity, DPM requires consumers to consider how similar the target product is to the other products in the external representation and then translate this perceived similarity into a distance. In this context, perceptual measurement is thus the degree to which a similarity-based product perception is translated into a distance in the external representation. A second important and related construct is the difficulty of the DPM task. A number of factors may reflect task difficulty, including the subject's ability to think in terms of the representation, whether the subject agrees with the distances among products in the representation, the subject's confusion with the task, and the ease or enjoyment with which the subject performs the task. Our conceptual model describes the effects that qualitatively different external representations and individual differences have on these two important constructs. The model, presented in Figure 3, explains the perceptual measurement and the task difficulty of a DPM as a function of two important distinguishing aspects of the external representation and three important individual differences, all of which are described below. 9

Insert Figure 3 Notice that task difficulty has a direct, negative effect on perceptual measurement. The remaining constructs affect perceptual measurement both directly and indirectly through task difficulty. The External Representation DPMs may be collected using qualitatively different representations, the most carmon of which are product spaces and trees (see Figure 1). A second difference of importance to marketing research is the categorical level or abstractness of the products involved. Products ccrpete at different levels of abstraction. At a brand-level, for example, Coke competes with Pepsi, while at a more abstract, category-level soft-drinks ccmpete with other thirst quenching beverages. These two important dimensions, the spatial versus treebased nature of the representation and the brand- versus category-level of the stimuli, should systematically affect perceptual measurement and task difficulty. Consider first the inherent differences between trees and spaces. The tree-like representations of hierarchical clustering and additive tree scaling provide a viable alternative (or cctpliment) to perceptual spaces. Such tree-like structures are very consistent with the featurebased representations that underlie many consumer products (Johnson and Fornell 1987) and are often better able to capture the perceptions of conceptual stimuli (Pruzansky, Tversky and Carroll 1982; Tversky and Hutchinson 1986). However, one of the major advantages of spatial representations is their intuitive appeal and accessibility to the human eye (Shepard 1972). From a naive subject's standpoint, spatial distance 10

"as the crow flies" is an intuitive concept that may be easily related to their perception of similarity. Thinking of product similarities as distances within or along the paths of a tree may not be so natural. As a result, we predict that using spaces increases perceptual measurement relative to using trees. The direct positive relationship frcm the spatial (versus tree) nature of the representation to perceptual measurement in Figure 3 captures this relationship. The naturalness and intuitive appeal of spatial representations also suggests that spaces make for an easier judgment task. The direct negative relationship from the spatial nature of the representation to task difficulty captures this relationship. The second important aspect of the external representation modeled here is the category- versus brand-level of the stimuli involved. Following Howard (1977), consumers proceed by a process of grouping and distinguishing products into a hierarchy of categories ranging frmn relatively abstract, superordinate product categories, to more subordinate categories, to brands (see also Sujan 1985 and Johnson and Fornell 1987). There is an important difference between more traditional product categories and brands which may significantly impact both task difficulty and perceptual measurement. Recent research suggests that traditional product categories, such as soft-drinks or candy bars, are equivalent to the basic-level categories studied in psychology (Johnson and Fornell 1987). What distinguishes basic-level categories from other categories is the high degree of inclusiveness or similarity of their members (defined as the ratio of ccmnon to distinctive features; Rosch 1975, Rosch et al. 1976). This suggests that traditional product categories are well-defined and 11

perceptually distinct relative to brands. Consider, for example, a subject who finds it rather straightforward just where to place a carrot relative to other snack foods in a space or tree, yet is unsure just where to place Snickers relative to other candy bars. We predict that using perceptually distinct category-level stimuli will increase perceptual measurement and decrease task difficulty relative to using brands. Both relationships are captured in Figure 3. Individual Level Knowledge and Product Preference The model incorporates the effects of individual differences in knowledge and preference on perceptual measurement and task difficulty. Regarding knowledge, we distinguish between the consumer's knowledge of the target product and their knowledge of the entire set of products involved in the task. In Figure 3, product knowledge captures the former while stimulus set knowledge captures the latter. The fact that respondents concentrate on the target product suggests it be treated separately. Target product knowledge is necessary to understand and judge the object of comparison, while stimulus set knowledge provides an understanding of the reference group or context of the perception. One straightforward prediction is that target product knowledge and general stimulus set knowledge both increase perceptual measurement and decrease task difficulty. The positive relationships from the knowledge constructs to perceptual measurement and corresponding negative relationships to task difficulty capture these predictions. Preference for the target product should also affect task difficulty and perceptual measurement. Product preference has the potential to bring nonperceptual factors into play. For example, the 12

uniqueness and positive affect associated with a preferred product (Carpenter and Nakamoto 1988) may make a direct perception of similarity difficult. Therefore, we predict that product preference decreases perceptual measurement and increases task difficulty (see Figure 3). For completeness, the model incorporates the natural relationships involving the individual difference constructs. Following Howard (1977), category-level knowledge develops prior to brand-level knowledge. As a result, our product category knowledge should, in general, exceed our knowledge of brands (Sujan 1985). The result is a direct positive relationship from the category versus brand nature of the stimuli to stimulus set knowledge. Stimulus set knowledge should, in turn, positively affect target product knowledge. The greater our knowledge of a set of product stimuli, the greater should be our knowledge of any one member of that set. Meanwhile, both product knowledge and stimulus set knowledge should affect product preference. The more knowledge we accumulate regarding the target product, the more likely we are to prefer and 2 purchase that product. Thus we posit a positive relationship from product knowledge to product preference. Similarly, the greater our knowledge of the other products in the stimulus set, the lower should be our relative preference for the target product. Thus we posit a negative relationship from stimulus set knowledge to target product preference. These are the predominant relationships among the constructs. Notice that the model posits general positive effects for spaces versus 2. The directionality of this relationship might be reversed if past preference were involved. That is, while current knowledge affects current preference, past preference affects current knowledge. 13

trees, categories versus brands, stimulus set knowledge and product knowledge on perceptual measurement. These positive effects are both direct and indirectly mediated by task difficulty. Product preference has a similar general negative effect on perceptual measurement. An empirical study was conducted to test this conceptual model and examine the nomological validity of direct perceptual measurement. EMPIRICAL Sru Study Design The study was conducted in two phases. The external representations were built in phase one. These representations were used in phase two to collect direct perceptual measures and test the model. The space versus tree and category versus brand nature of the external representations were manipulated, while most of the remaining endogenous constructs were modeled using multiple indicators. In phase two, each subject was asked to indicate their perception of a single product within three qualitatively different representations, a spatial representation derived from MDS, and two tree-based representations, one derived from hierarchical clustering and one derived from additive tree scaling. All of the subjects were asked to provide verbal protocols while they performed the tasks (Ericsson and Simon 1980). Information was also collected regarding the subjects' kncwledge, preferences, and task difficulty. Phase One In phase one a convenience sample was asked to provide proximity judgments for products from one of five possible stimulus sets. Two of these sets involved brands from the same product category: soft-drinks and candy bars. Three stimulus sets involved different basic-level 14

product categories from the same superordinate-level category: beverages, snacks, and lunch products. Each set contained twelve product alternatives requiring subjects to make 66 paired-ccmparisons. A total of 24, 24, 24, 24, and 27 subjects (n=123) rated the soft-drink, candy bar, beverage, snack food, and lunch product stimuli respectively. Each pair of alternatives was rated on a scale frcm 0 (very dissimilar) to 10 (very similar). Half of the subjects in each group rated the 66 pairs in one random order and the other half rated the same 66 pairs in the reverse order. The judgments were pooled for each stimulus set and scaled using three qualitatively different techniques: multidimensional scaling (MIDS) in two dimensions, hierarchical clustering (HCS) using an average linkage method (Johnson 1967), and additive tree scaling via ADDTREE (Sattath and Tversky 1977). The MDS, HCS, and ADDTREE solutions for candy-bars are those shown in Figure 1. The fifteen different representations across the five different stimulus sets all provided reasonably good fits to the input data (Kruskal's stress for the solutions ranged from.04 to.13). Phase Two In phase two the MDS, HCS and ADDTREE representations were used to collect DEMs on a new group of subjects. Direct perceptions were collected for ten products overall, tw from each stimulus set. The products included Pepsi Cola and Diet Orange Crush frcan the soft-drinks, Snickers and Reece's Peanut Butter Cups from the candy bars, Fruit Juice and Ice Cream Soda frcm the beverages, Popcorn and Cheese fromn the snack foods, and Pizza and Carrot frcm the lunch products. The DPM procedure outlined earlier was followed for each product/space and product/tree 15

combination. For example, Pepsi was erased frcn the MDS solution and the Pepsi branch was removed fran the HCS and ADDTREE representations. (Again, the tree representations were cosmetically altered so as not to reveal where a branch had been removed.) A total of 198 subjects were asked to provide DPMs (approximately twenty subjects per product). Each subject indicated their perception of the target product within each of the three possible external representations. Given the possibility of carryover effects, the order of DPM collection was systematically rotated. Approximately ten subjects in each product group provided the space DPM followed by the tree DPMs while this space/tree order was reversed for the remaining subjects. Within each space/tree ordering, the order of the HCS and ADDTREE representations was also counterbalanced, resulting in four randan order conditions. Phase two was conducted using adult subjects recruited in a suburban mall of a major metropolitan area. Subjects were paid for their participation. Using a pencil and paper format, subjects first rated their confidence in evaluating each of the twelve alternatives frcm the stimulus set on a scale fron 0 (not at all confident) to 10 (very confident). Subjects then rated their attribute knowledge of each product on an absolute, twenty-one point scale. The scale reflects the subjects' knowledge and understanding of the products' attributes and functions, ranging from 0 (no knowledge of the product), to 5 (knowing only what the product may be used for), to 10 (knowing how to use the product but not having considered its details and functions), to 15 (having thoughtfully considered the product's details and functions), to 20 (knowing every aspect of the product and its uses). This absolute 16

knowledge scale (adapted from Johnson 1984) allows for comparisons across product categories. Each subject then rank ordered their preference for the twelve alternatives. Subjects were then shown to an interview room where an experimenter individually ran each subject through the DEM tasks and collected their verbal protocols. After collecting the three DPMs for the target product, the subject was asked to cmrplete sane final questions regarding each of the MDS, HCS, and ADDTREE tasks. Subjects were asked to rate each separate task on seven point agree-disagree scales regarding the following statements: (1) I found it easy to think in terms of the map (tree), (2) I generally agreed with the distances among products in the map (tree), (3) I found the task confusing, (4) placing the product in the map (tree) was easy, and (5) placing the product in the map (tree) was fun. Verbal Protocol Coding and Analysis Of the 594 task protocols collected in the study, some were excluded frmn the analysis. Following standard verbal protocol procedures (Ericsson and Simon 1980), the experimenters were instructed only to prompt people to "think aloud" and, when quiet, to ask them "what are you thinking." In some unforeseen cases, an experimenter unduly coached or directed a subject and this data was excluded from further analysis. Equipnent problems also caused the loss of some protocols. The end result was a total of 448 usable protocols and 535 usable DEMs. Two naive judges divided the protocols into separate meaningful statements. Two judges (one carry over and one new, naive judge) then independently coded these statements into six general categories: (1) 17

attribute, brand, or category based similarity/dissimilarity statements (e.g., "it is similar to....," "it tastes like...."), (2) distance statements (e.g., "it goes between...," "it should be close to...."), (3) positive/negative statements regarding the products and their placement, (4) statements eliciting satisfaction or dissatisfaction with the external representation, (5) statements regarding product usage occasions, and (6) miscellaneous statements and task impressions (e.g., recalling past experiences, statements of understanding, pondering, etc.). The judges agreed in classifying over 89% of the statements and the discrepancies were resolved by discussion. After coding the statements in each protocol, the judges independently counted the number of products mentioned. The judges' agreement here was over 95% and discrepancies were again resolved by discussion. The resulting protocol code revealed that out of the 2,170 total protocol statements, there were 1,203 (55.4%) similarity statements and 401 (18.4%) distance statements. Of the remaining statements, only 96 (4.4%) were positive/negative statements regarding products, 55 (2.5%) revealed satisfaction/dissatisfaction with the external representations, 54 (2.4%) were statements regarding product use occasions, and 361 (16.6%) were in the miscellaneous category. Of the miscellaneous statements, by far most were subjects repeating their reasons for or justifying their perceptions. Other miscellaneous statements included expressions of pondering, fun, learning, recalling past experiences, displeasure, difficulty, and confusion. Overall, the similarity and distance statements comprised 73.8% of the total protocols statements. This supports the general similarity-distance orientation of the DPMs. 18

Model Estimation Task difficulty is treated as a latent construct, with each subject's responses to the five task difficulty statements of each DPM serving as reflective indicators (D1 - D5). Three formative indicators obtained from each DEM protocol measure latent perceptual measurement: the number of similarity statements (M1), the number of distance statements (M2), and the number of products considered (M3). Recall that perceptual measurement requires subjects to consider the products involved, in terms of similarity, and translate their perception into a distance, supporting the use of formative indicators. The model includes a dichotcmous space versus tree variable (S1). Preliminary analysis of variance models revealed significant contrasts between the spaces and the two trees, but not between the trees themselves. This allows us to collapse the three different representations into the tw-level space versus tree construct. A similar two-level variable captures the category- versus brand-level of the stimuli (C1). Stimulus set knowledge, which captures each subject's knowledge of all the products involved in the task, is modeled reflectively by two indicators, the subject's average self-rated confidence in evaluating the products in the set, and the subject's average self-rated attribute knowledge for the products (K1 and K). Product knowledge, which captures the subject's knowledge of the target product, is also modeled reflectively by two indicators, the subject's self-rated confidence in evaluating the target product, and the subject's self-rated attribute knowledge for the target product (K3 and K4). Preference is modeled 19

reflectively by the subject's reverse rank order preference for the target product (P1). Given the nature of the model and data, the model was estimated using partial least squares (Fornell 1988; Fornell and Bookstein 1982; Wold 1982). The construct indicators were scaled such that all of the loadings should be positive. Model Results The indicator loadings and structural coefficients are presented in Figure 4. Insert Figure 4 All of the indicator loadings were relatively large and positive which 3 supports a sizable amount of valid variance in the measurement. There is considerable support for the hypothesized causal structure. The model explains 85% of the covariance among the latent variables. (The measurement model explains 56% of the covariance among the measurement 2 variables.) The R 's for the endogenous constructs, task difficulty, perceptual measurement, stimulus set knowledge, product knowledge, and product preference, equaled.08,.09,.01,.29 and.26 respectively. This reveals that there was very little variance in stimulus set knowledge. Using very different target products from the different stimulus sets did, however, produce sizable variation in product knowledge. 3. The relative large loading for distance statements relative to similarity statements and products considered for the perceptual measurement construct is, in hindsight, not surprising. The subjedts' ability to translate their perceptions into a distance is the key to the methodology. 20

Overall, fourteen of the fifteen structural coefficients are in the predicted direction. Task difficulty had a direct, negative effect on perceptual measurement. As predicted, the use of spatial representations decreased task difficulty and increased perceptual measurement. This is consistent with spatial representations being a more intuitive and natural way to directly collect perceptions. The results also support the predicted positive effect of using product category stimuli. Using categories as opposed to brands increased perceptual measurement and decreased task difficulty. This is consistent with the relatively stable and distinctive nature of basiclevel product categories relative to brands. The use of categorical stimuli increased stimulus set knowledge, and the hypothesized relationships among the knowledge and preference constructs were well supported. Set knowledge positively affected product knowledge which, in turn, positively affected product preference. Set knowledge, meanwhile, had a direct negative effect on product preference. Finally, the results reveal individual difference effects on task difficulty and perceptual measurement. Although stimulus set knowledge had small predicted effects on these constructs, given the relatively 2 low R for set knowledge, we focus on the more meaningful results involving product knowledge and preference. As predicted, product knowledge had a negative effect on task difficulty and a positive effect on perceptual measurement. This is consistent with the notion that product knowledge improves a consumer's ability to understand and provide a DPM. The predicted positive relationship between product preference and task difficulty also materialized. It appears that 21

preference for the target product makes its DPM more difficult. Not predicted was the observed positive effect for product preference on perceptual measurement. A possible explanation of this unexpected result is that preferred products sparked greater interest or motivation on the part of our subjects, increasing perceptual measurement. Overall the model demonstrates the general positive effects of spatial representations, product category stimuli, and knowledge on perceptual measurement. These effects are both direct and mediated by task difficulty. The systematic nature of the results supports the conceptual model and the nomological validity of direct perceptual measurement. It is interesting, however, that the structural coefficients involving task difficulty and perceptual measurement are not drastically large. We shall explore this point further in the discussion section of the paper. DPM Product Positions One final analysis was performed in order to examine the sensitivity of the output of the methodology to individual differences in perceptual measurement, task difficulty, knowledge, and product preference. To do so we compared the subjects' direct perceptions with the products' positions prior to their elimination from a representation. Using the products' initial, analytically derived positions as a benchmark is a heuristic way of assessing the face value of a DEM (Johnson and Horne 1988). We measured the distance between each DPM and the target product's original position in millimeters. (Distance in the trees was measured frco where the subjects connected their branch for the target product to 22

the connection of the target product's original branch.) These distances were compared with each of the following variables: (1) Average stimulus set knowledge (2) Average stimulus set confidence (3) Target product knowledge (4) Target product confidence (5) Target product rank order preference (6) Number of similarity oriented protocol statements (7) Number of distance oriented protocol statements (8) Number of products considered in the protocol (9) The five indicators of task difficulty Given the interrelationships among many of these variables, separate linear models were estimated using each of the covariates listed above to explain the distance between the DEMs and the products' original positions. The results reveal significant negative correlations (p<.05) between the distance measures and the number of similarity oriented statements (r=.14), the number of distance oriented statements (r=.ll), and the number of products overtly mentioned in the protocols (r=. 11). In other words, the larger our perceptual measurement construct, the closer were the subjects' DEMs to the target products' original positions in the external representations. The only other significant relationship was an increase in distance with task confusion (r=.13, p<.01). SLMMARY AND DISCUSSION Direct perceptual measurement provides marketing researchers with a viable supplement to the more traditional methods for studying product perception. Its advantages are realized in those cases where the data requirements of the traditional methods are limiting or prohibitive. We set as our goal to develop and test a conceptual model of the proposed methodology. The model demonstrates how both the representation used to collect perceptions and individual differences in knowledge and 23

preference affect direct perceptual measurement. Our empirical results generally support the model and the norological validity of the methodology. The results also provide insight into the method's application. One of our more interesting findings concerns the inherent advantage of using spatial representations to directly assess perceptions. Although tree-scaling offers many advantages and is showing increased popularity in marketing and consumer research (see Arabie, Carroll, DeSarbo, and Wind 1981; Johnson and Fornell 1987; Srivastava, Leone, and Shocker 1981), spatial representations are very intuitive, especially to the naive consumer. Using spatial representations increased perceptual measurement and decreased task difficulty relative to using product trees in our study. There are similar advantages to using category-level stimuli. Product categories, being similar to the basic-level categories studied in psychology, are perceptually distinct and stable relative to brands. As a result, when our subjects were presented with categories rather than brands, perceptual measurement increased and task difficulty decreased. Finally, our results suggest that perceptual measurement increases with the subject's knowledge of the target product. Although the model was generally supported, recall that the structural coefficients were not extremely large. There are same important considerations that put the magnitude of the effects in perspective. First, recall the predominance of similarity and distance related statements in the verbal protocols. All of the different DPMs, whether they involved trees or spaces, categories or brands, likely had a basic similarity-distance orientation. Second, while there were 24

differences in task difficulty across the DPMs, all in all there were no major difficulties. To illustrate, the consumers' average responses to the task difficulty questions are reported in the Table. Insert Table Notice that while all five questions support the relative ease of spaces over trees and categories over brands, none of the representations or stimuli posed a particularly difficult task for our consumers. Overall, the protocol results and the task difficulty responses suggest the existence of ceiling effects for both the perceptual measurement and task difficulty constructs. As a result, any advantage for spaces over trees or categories over brands is naturally limited. Meanwhile, the relatively small effects for product knowledge on perceptual measurement and task difficulty may be due, in part, to the relatively ccrmon nature of the stimuli used in the experiment. The effects of product knowledge might have been greater if less acmnon stimuli had been used on which consumers were more likely to vary in knowledge (e.g. personal ccmputers). Finally, it is important to remember that the methodology was designed with particular data collection problems in mind. As DEM requires a conventional perceptual representation as a starting point, it can only augment existing methods. We hope that DEM will, in fact, extend the use of perceptual mapping and tree scaling to problems that may not have previously lent themselves to such applications. As with other perceptual measures, DPMs may be subject to a variety of contextual biases. It will be important for future research to explore 25

these biases in order to refine the methodology and interpret its output. 26

FIGURE 1 ALTERNATIVE REPRESENTATIONS OF LEADING CANDY BARS Figure 1A: Multidimensional Scaling Solution * Three Musket rs e M&M Plain * Mar's Bar ~ M&M Peanut Milky Way * * Snickers * Hershey's Plain * Hershey's Almond * Reece's Peanut * Nestle's Crunch Butter Cup Twix Carmel * Kit Ket Figure 1B: Hierarchical Clustering Scheme Mar's Bar Three Musketeers Milky Way Snickers M & M Plain M & M Peanut Reece's Peanut Butter Cup Hershey's Almond Hershey's Plain,i ~,________ [Nestle's Crunch Twix Caramel Kit Kat Figure 1C: Additive Tree Kit Kat Twix Caramel Snickers Milky Way Three Musketeers Mars Bar Nestle's Crunch.j --- —- *- ~Hershey's Plain Hershey's Almond Reece's PBC M&M Peanut M&M Plain

FIGURE 2 SAMPLE DRI OUTPUT FOR SNICKERS Figure 2A: Multidimensional Scaling Solution * Three Musketeers * M&M Plain A/ fMar's Bar A A t L~~~MarA~ Bar * M&M Peanut Milky Way,A A * Hershey's Plain aO * Hershey's Almond L\A A~^~~~~~~ * Reece's Peanut * Nestle's Crunch Butter Cup A * Twix Carmel * Kit Kat A Figure 2B: Hierarchical Clustering Scheme Mar's Bar Three Musketeers Milky Way M & M Plain i[ ____ M & M Peanut Reece's Peanut Butter Cup Hershey's Almond Hershey's Plain A, __________ ~Nestle's Crunch Twix Caramel _ Kit Kat Figure 2C: Additive Tree Kit Kat _l ^~-~~ ---- ' ------ -Twix Caramel 4 A |T Ih --- Milky Way — * -------- *-~^ -— ' - ~ Three Musketeers ' a, 'Mars Bar N --- Nestle's Crunch Hershey's Plain -A ---- l ~~~~~Hershey's Almond Ar" --- —--- - ~~~~~~~~Reece's Peanut Butter Cup __ -t1_ __LZZL_ ---- M&M Peanut M&M Plain A = Direct Perceptual Measures A = Target Product Original Location 0 = Average DPM for Spatial Representation

FIGURE 5 PERCEPTUAL MEASUREMENT MODEL

HOUFRE 4 PLS MODEL. RESULTS -.32

TABLE TASK DIFFICULTY DIFFERENCES ACROSS DPMs Representation Level of Abstraction Dependent Variable MD6 HCS ADDTREE Brands Categories Easy to think in terms of 2.30 2.67 2.62 2.86 2.33 Representation Agreed with the distances in 2.49 2.62 2.71 2.69 2.56 Representation Found the task Confusing 5.53 5.34 5.26 5.07 5.56 Placing the product was 2.12 2.61 2.43 2.65 2.23 easy Placing the product was 2.87 3.10 3.07 3.16 2.91 fun * Response scale range of 1 (Agree) to 7 (Disagree) 31

REFERES Arabie, Phipps, J. Douglas Carroll, Wayne DeSarbo, and Jerry Wind (1981), "Overlapping Clustering: A New Method for Product Positioning," Journal of Marketing Research, 18 (August), 310-317. Bagozzi, Richard P. (1980), Causal Models in Marketing, New York: John Wiley & Sons. Carpenter, Gregory S. and Kent Nakamoto (1988), "Consumer Preference Formation and Pioneering Advantage," Research Working Paper Series, Columbia Business School, New York, NY, 10027. Carroll, J. Douglas and J. J. Chang (1973), "A Method for Fitting a Class of Hierarchical Tree Structure Models to Dissimilarities Data and its Application to Some 'Body Parts' Data of Miller's," Proceedings of the 81st Annual Convention of the American Psychological Association, 8, 1097-1098. Cooper, Lee G. (1983), "A Review of Multidimensional Scaling in Marketing Research," Applied Psychological Measurement, 7 (Fall), 427-450. Cronbach, Lee J. and Paul E. Meehl (1955), "Construct Validity in Psychological Tests," Psychological Bulletin, 52 (4), 281-302. DeSarbo, Wayne S. and Donna L. Hoffman (1987), "Constructing MDS Joint Spaces from Binary Choice Data: A New Multidimensional Unfolding Threshold Model for Marketing Research," Journal of Marketing Research, 24 (February), 40-54. and Vithala R. Rao (1984), "GENFOLD2: A Set of Models and Algorithms for the GENeral UnFOLDing Analysis of Preference/Dominance Data," Journal of Classification, 1 (Winter), 147-186. Dong, Hei-Ki (1983), "Method of Ccmplete Triads: An Investigation of Unreliability in Multidimensional Perceptions of Nations," Multivariate Behavioral Research, 18 (January), 85-96. Ericsson, K. Anders and Herbert A. Simon (1980), "Verbal Reports as Data," Psychological Review, 87 (3), 215-251. Fornell, Claes (1988), "The Blending of Theoretical and Empirical Knowledge in Structural Equations with Unobservables," in Herman Wold (ed.), Theoretical Empiricism, New York: Paragon House. and Fred L. Bookstein (1982), "Two Structural Equation Model: LISREL and PLS Applied to Consumer Exit-Voice Theory," Journal of Marketing Research, 19 (November), 440-452. Green, Paul E. (1975), '"Marketing Applications of MDS: Assessment and Outlook," Journal of Marketing, 39 (January), 24-31. 32

and Frank J. Carnone (1970), Multidimensional Scaling and Related Techniques in Marketing Analysis, Boston, MA: Allyn and Bacon. Hauser, John R. and Frank S. Koppelman (1979), "Alternative Perceptual Mapping Techniques: Relative Accuracy and Usefulness," Journal of Marketing Research, 16 (November), 495-506. Howard, John (1977), Consumer Behavior: Application of Theory, New York: John Wiley & Sons. Johnson, Michael D. (1984), "Consumer Choice Strategies for Comparing Noncanparable Alternatives," Journal of Consumer Research, 11 (December), 741-753. and Claes Fornell (1987), "The Nature and Methodological Implications of the Cognitive Representation of Products," Journal of Consumer Research, 14 (September), 214-228. and David A. Horne (1988), "The Contrast Model of Similarity and Comparative Advertising," Psychology and Marketing, 5 (3), 211-232., Donald R. Lehmann, and Daniel R. Horne (1988), "The Effect of Fatigue on Judgments of Interproduct Similarity," Working Paper #579, Division of Research, The University of Michigan, School of Business Administration, Ann Arbor, MI, 48109. Johnson, Stephen C. (1967), "Hierarchical Clustering Schemes," Psychanetrika, 32 (3), 241-254. Kruskal, J. B. (1964), '"Multidimensional Scaling by Optimizing Goodness of Fit to a Nonmetric Hypothesis," Psychanetrika, 29 (1), 1-27. Malholtra, Naresh K. (1987), 'Validity and Structural Reliability of Multidimensional Scaling Results," Journal of Marketing Research, 24 (May), 164-173. Peter, J. Paul (1981), "Construct Validity: A Review of Basic Issues and Marketing Practices," Journal of Marketing Research, 18 (May), 133-145. Pruzansky, Sandra, Amos Tversky, and J. Douglas Carroll (1982), "Spatial Versus Tree Representations of Proximity Data," Psychanetrika, 47 (1), 3-24. Rosch, Eleanor (1975), "Cognitive Representation of Semantic Categories," Journal of Experimental Psychology: General, 104 (September), 192-233., Carolyn B. Mervis, Wayne D. Gray, David M. Johnson, and Penny Boyes-Braem (1976), "Basic Objects in Natural Categories," Cognitive Psychology, 8 (July), 382-439. 33

Sattath, Shmuel and Amos Tversky (1977), "Additive Similarity Trees," Psychcmetrika, 42 (3), 319-345. Shepard, Roger N. (1962), "The Analysis of Proximities: Multidimensional Scaling with an Unknown Distance Function. I and II," Psychanetrika, 27 (2), 125-140 and 219-246. (1972), "Introduction to Volume I," in R. Shepard, A. Rcmney, and S. Nerlove (eds.), Multidimensional Scaling: Theory and Applications in the Behavioral Sciences, Volume I: Theory, New York: Seminar Press, 1-20. and Phipps Arabie (1979), "Additive Clustering: Representation of Similarities as Combinations of Discrete Overlapping Properties," Psychological Review, 86 (2), 87-123. Srivastava, Rajendra K., Robert P. Leone, and Allan D. Shocker (1981), "Market Structure Analysis: Hierarchical Clustering of Products Based on Substitution-in-Use," Journal of Marketing, 45 (Summer), 38-48. Sujan, Mita (1985), "Consumer Knowledge: Effects on Evaluation Strategies Mediating Consumer Judgment," Journal of Consumer Research, 12 (June), 31-46. Summers, John 0. and David B. MacKay (1976), "On the Validity and Reliability of Direct Similarity Judgments," Journal of Marketing Research, 13 (August), 289-295. Tversky, Amos and J. Wesley Hutchinson (1986), "Nearest Neighbor Analysis of Psychological Spaces," Psychological Review, 93 (1), 3-22. Wold, Herman (1982), "Systems Under Indirect Observation Using PLS," in Claes Fornell (ed.), A Second Generation of Multivariate Analysis: Methods, New York: Prager, 325-347. 34

ABSTRACR The paper describes a methodology for the direct measurement of product perceptions. The method uses the spatial or tree-like representations of traditional similarity scaling methods as data collection devices. Consumers record their perception of a product directly within an existing tree or space. A conceptual model of the methodology is developed and tested. The results generally support the model and the ncrological validity of direct perceptual measurement. * Submitted to the Journal of Marketing Research. Do not quote or reproduce any portion of this manuscript without permission. ** Michael D. Johnson is Associate Professor of Marketing at the University of Michigan's School of Business Administration, Ann Arbor, Michigan, 48109, and David A. Horne is Associate Professor of Marketing at the School of Business Administration, California State University, Long Beach, California, 90840. The authors thank Donald R. Lehmann and Claes Fornell for their cmanents on an earlier draft of the paper. The financial support of the Marketing Science Institute and the Executive Education Research Fund of the University of Michigan's School of Business Administration is also gratefully acknowledged.