3US. ADM. IB^Y I RESEARCH SUPPORT UNIVERSITY OF MICHIGAN BUSINESS SCHOOL FEBRUARY 1996 MAINTAINING REPRESENTATIVE NATIONAL SAMPLES OF CUSTOMERS OF 200 COMPANIES WITHIN THE SURVEY FOR THE AMERICAN CUSTOMER SATISFACTION INDEX (ACSI) WORKING PAPER #9690-01 BARBARA EYERITT BRYANT UNIVERSITY OF MICHIGAN JANICE A. BROWN MARKET STRATEGIES, INC. ANN SULLIVAN MARSDEN MARKET OPINION RESEARCH - PACE

L- ' *:-.-."..' ~..,: '. *,... a...!~i!5:3 ^ '". -, *..*..*' '... ** *..... h - - - -." ":'e Lss i ' "':.,,......... *-.;s~~~~~~~~~~~~~ *.;. '>..''" ' *: -'." *,,,..'...". 7.".~."**.:.*.;.-.-5 '.:.:....... " ":".*."*..* ' ' *: ', ','..', *.L'.,'s.. '.:..4. ': '" '.;'-. '.... ' "*':'"::': " " '.... '* ~ ~~~~~~~~~~~~~~~~.'.-.',. -.,~~~~~~~~~~~~~.... '... k~~ ~,1!ii!i..;-... '"1 iii iti11 ".i ~-;~@e IAk S1~ z......:.... ~.. '..........,; " '::-~~ A^ -

-)' i.1 I I I C) ^-..,.. ~...: *.... O.X ~.:..-..~.....:.:...t.. m.. ': ',,Ee...O *.' "...-..:...........:.....'....... i::y.^..^.. '. i!?-'',.-:::'.,,+ub5 Zz, * <. 't*@-v~zb';'lmmm. '. ^^ySc^at^^A it^^^^^ '.^l.OS {r''SeiS^^^;^.... ',?(.......~..:, ~,,~...~.,:..-,..X... -,.,,~. @ -':.... * r. "'~-;'.' ~ '.:.%'.- '.' r -.u ',...... m m I.:' * *..:.*.' '**>.. '.. '.. ': ".': i.-,,_,.!:? i?:;.. m I mmb' mm mm'.-N *~~~~~~~~~~~~~~~~o:.....: ^ ^.*''.^ I.?-.. ****^-<-. —.. *!:^ <,<-:..:'.^:.~,,*..r-f*...-..J':5 *****;^^~~~~~~~~~~~~~~~~~~~~~....,...... ~.'. 3..... 'r' ~. * *. L *' *..-:ng4.*; *.a -.. w d;t' I p-sf.1 A - I,., 2, - -- -- - -.1-,;I*,..- -. - _IZ- - -

Maintaining Representative National Samples of Customers of 200 Companies Within the Survey for the American Customer Satisfaction Index (ACSI) Barbara Everitt Bryant University of Michigan Business School Janice A. Brown Market Strategies, Inc. Anne Sullivan Marsden Market Opinion Research-PACE for the Midwest Association of Public Opinion Research Chicago, Illinois November 17, 1995 Have you ever tried to maintain 200 national, representative, random-digit-dial (RDD) samples of customers simultaneously selected from the same sampling frame? That's the sampling and data collection challenge for the American Customer Satisfaction Index, an economic indicator of customer satisfaction with the quality of goods and services purchased and used by U.S. households. I II I FIGURE 1: WELCOME TO THE AMERICAN CUSTOMER SATISFACTION INDEX t I II 1.l L -. -.- "I'r - -An- c~ —r —imm-'

I * ACSI, which is cosponsored by the University of Michigan Business School and the American Society for Quality Control, is a uniform, cross-industry index now completing its second year. ACSI is based on econometric modeling of the satisfaction of customers of 200 companies in 34 industries in seven sectors of the economy which produce almost 38% of the gross domestic product. FIGURE 2: ACSI: NATIONAL ECONOMY, SECTORS AND INDUSTRIES The ACSI uses a multi-equation model, tested in Sweden since 1989, with pretests in the U.S. in 1993 and 1994, and now used for two years here. FIGURE 3: THE ANALYSIS MODEL The model describes the relationships among six key constructs, each of which is a weighted average of two or three evaluation questions asked customers: (1) customer expectations, (2) customer perceptions of quality based on experience, (3) customer perceptions of value, (4) customer satisfaction, the ACSI, based on questions on overall satisfaction, whether or not the goods or service fell short of, met, or exceeded expectations, and comparison of the good or service received to an ideal one. ACSI is related, in turn, to its outcomes-complaints, and customer loyalty. Loyalty is measured by questions on intent to repurchase and on several price tolerance questions. 2 I,-l-~., _. ~ ~..4, I VNV-.I m, I -, mi. 1l- -... I

The input to the separate modeling of satisfaction for each of the 200 companies comes from interviews with the customers of those companies. As you can see from the model, there are six blocks of substantive questions, 17 in all, plus eight demographic questions. FIGURE 4: THE QUESTIONS The challenge is not the interview, which is a relatively straight forward one, but the screening to actual customers and the simultaneous management of 200 samples in a cost effective manner. Selection and maintenance of so many samples at once is made more challenging because the incidence of purchase and use varies dramatically by industry and, within each industry, the market shares of the measured companies have a wide range. On top of that, ACSI considers as customers only those who have both purchased and used a good or service within a specific time period —time periods which range from three years for major durables to past week for reading a newspaper or currently having an account in one's own name for a bank. Conducting customer surveys for the American Customer Satisfaction Index requires screening randomly selected adults in sample households to find customers of a company such as JVC, which has about 1.3% of the market of those who bought a TV or VCR in the past three years, to the U.S. Postal Service with near universal daily use. ACSI screens for customers in an industry such as the cigarette industry. Smoking is down to 28% of the 21 year old and I 3 _...

I over population and within that industry the need is to identify customers of three companies with market shares at the 56%, 18%, and 10% level. Or the problem is identifying past week readers of Gannett Company newspapers. Gannett has both the large national circulation USA Today but over 100 other newspapers in specific geographic markets, some of them quite small. ACSI cannot use a single probability sample and let the customers fall as they do!in the population. That would produce an enormous number of interviews for the U.S. Postal Service the IRS, Campbell Soup Company, General Mills, or Procter & Gamble Company in order to get samples large enough to do modeling for Compaq Computer, JVC, or BMW. BMW is not exactly the people's car, nor is Volkswagen in the U.S. — although the German name translates out to the "people's vehicle." Therefore, a quota of interviews is set for each company. There is also a major complication to customer screening and identification when one is measuring companies like General Mills, Sara Lee, and Procter & Gamble-their customers don't know the names of the companies that produce the products they use! When a customer is asked what breakfast cereals he or she purchased and consumed in the past month, he or she comes up with a name like Cheerios. Cheerios is made by General Mills. And what woman names Sara Lee as the source of her Hanes pantyhose, or connects Duz, Era, Joy or Ivory to Procter & Gamble? Relatively few know the companies behind major consumer products. 4 -..-. --,.-.. - - I...... - - - ----

These were the sampling and data collection challenges the National Quality Research Center at the University of Michigan Business School laid out in a Request for Proposal in early 1994. The plan was to conduct interviews in all 200 samples for seven economic sectors simultaneously for the first year baseline survey, then in subsequent years, break the sample into four quarterly samples updating 1-2 sectors per quarter. Those in a business school believe in free market competition and the Request for Proposal was a marketing challenge. Proposers were judged on quality, cost efficiency, the capability of managing a project of this scale and complexity, and willingness to work with the business school in developing a practically implementable final design for screening respondents to identify customers for the many products of the large number of organizations specified for measure. The project design required some unusual CATI (computer-assisted-telephone-interviewing) system programming. Market Strategies, Inc. of Southfield, Michigan, came out ahead in what was a fairly close competition-the University of Michigan's Survey Research Center chose not to bid. Today, as the third year of the ACSI starts, it utilizes a sampling and field design which includes 48 replicate national samples annually, geographic regional oversamples, a dual frame of RDD plus list samples for the automotive industry, an unusual screening program which takes into account the incidence of use of the products and services of each industry and the market shares of measured companies within it, and allows for accumulation of short interviews with the customers of non-measured I II i II I 5.1

I companies into an "all other companies" category within a measured industry. Finally, a custom-designed CATI program, the ACSI Brand Extension Dictionary, links over 4,000 brands and brand extensions with their parent companies, assigning an interview to fulfill the company quota but inserting in the questionnaire used with the respondent the brand that customer names.. Let's run thorough the main features of the sampling, screening, customer identification, and interviewing program. The basis is RDD (random-digit-dial) sampling, but this is clearly a program "On Beyond RDD." FIGURE 5: RANDOM DIGIT DIAL METHODOLOGY-NATIONAL 48 REPLICATES For the baseline year, 1994, when all industries and companies were interviewed at once, a very Iarge —200,000 plus household —sample was drawn and broken into 48 national replicates. Currently, two sectors and approximately one-fourth of companies are updated each quarter using 12 replicates. That way interviews about high incidence goods and services can be completed using one or two replicates while many successively used samples are needed to fill the interview quotas for products with low incidence of use from companies with small market shares. ACSI measures some industries in which specific companies only operate in selected geographic areas. 6 i.24 - 0 r.

I FIGURE 6: ACSI SAMPLE DESIGN: Geographically Targeted RDD Sample For the automotive industry only, customer satisfaction is measured for 19 companies or major divisions of companies, the smallest one having only 1.5% market share. For that industry, RDD sampling is supplemented with the use of purchased vehicle owner lists. A dual sample frame is maintained throughout interviewing for this industry, with daily checks that the list sample is producing interviews with respondents with demographic characteristics similar to the RDD sample. FIGURE 7: ACSI SAMPLE DESIGN: Sample Used for Automobile Owners Although only 200 companies are measured and modeled in the ACSI, for industry customer satisfaction indices it is important to have representation of smaller companies. Therefore, three satisfaction questions are administered to those who have bought the industry products being measured, but not from the measured companies. FIGURE 8: ACSI "SHORT" QUESTIONNAIRES Sample and completed interview management is complex. Respondents are screened for the products and services of no more than 10 industries of the 34 measured. When interviews for an industry are completed, a new one is added to the screener. For 7 ___________________. _,^, -^r L 1, I I

I 1 *1 A I efficiency, respondents are screened for low use products first. They are interviewed about a maximum of three companies. Company interview quotas can be filled by interviews from different industries, for those companies which operate in more than one. Demographics are asked only once of an individual respondent. FIGURE 9: ACSI CATI SYSTEM —REQUIREMENTS FIGURE 10: ACSI CATI INTERVIEW: Sequence FIGURE 11: ACSI INTERVIEW DESIGN: Product Screening Order FIGURE 12: COMPANY AND BRAND INCIDENCE FOR ACSI Development and use of the Brand Extension Dictionary is one of the most innovative features of ACSI. Here's how it works. FIGURE 13: ACSI BRAND EXTENSION DICTIONARY FIGURE 14: ACSI BRAND EXTENSION DICTIONARY (2ND SLIDE) FIGURE 15: ACSI BRAND EXTENSION DICTIONARY: EXAMPLE With 200 customer interview samples to manage at once, University of Michigan Business School and Market Strategies agreed that MSI should institute close 8 L. ~ll N.~ ~..*

monitoring of the demographic profile of the completed interviews in each industry. Here are aggregated results for 1994 and the first quarterly update following the baseline. In actuality, however, the sample for each industry is watched daily during the field period and reported weekly to the University. FIGURE 16: SAMPLE COMPARISONS: ACSI CUSTOMERS FIGURE 17: SAMPLE COMPARISONS (2ND SLIDE) The topic of "Maintaining Representative National Samples of Customers of 200 Companies in the Survey for the American Customer Satisfaction Index," ends here. However, there is interest in what the ACSI shows. Customer satisfaction is modeled by a set of weighted ratings asked on 1-10 scales. The Index itself is transformed to a 0-100 scale. In the baseline study, released in October 1994, the national customer satisfaction index representing the economy stood at 74.5. FIGURE 18: 1994 ACSI=74.5 What could be determined initially was that U.S. consumers are more satisfied with manufacturing products than with services, and with private sector-supplied services than with government services. FIGURE 19: ACSI: Manufacturing, Services, Government 9 I * -" -.0 -1 I l-. - --- - I-." ...1. I I I - -11 - - -- -

I. I There was great variation in customer satisfaction with individual companies in the same industry. FIGURE 20: ACSI RANGE 1994: SELECTED INDUSTRIES Unhappily for both companies and the economy, customer satisfaction has declined in the second year of the ACSI, far more for services than for manufactured goods. FIGURE 21: CHANGES IN NATIONAL ACSI To date,, individual company scores have been available only to companies which participate as corporate sponsors of the ACSI. That changes as of December 1995, when the ACSI becomes a Fortune magazine list with satisfaction scores published for all the measured companies. However, the modeling detail, and the scores of constructs in the model, other than the ACSI, remain available only to corporate sponsors who receive detailed reports on their own industries and their competitors in these. ACSI uses a model which links customer satisfaction to customer loyalty, in the form of customer retention and price tolerance. Customer retention is a major determinant of profitability for a company. Hopefully, for individual companies and for 10 ,-r nr r~ ~~~~~~~~I~- - "11 - -l —,-W-' — ~ ~;.i c;; p,F L...i.".,.

the economy, ACSI will become a tool for improving customer satisfaction and with it customer retention. ** 11 I.-r, vp- mn R

-rA+ - 1 American Cus toner Sa tisfac Lion IndexIA American Customer Satisfaction Index (ACSI) ASQC ACSI is co-sponsorcd bly ASQ(C ai(I tlie University of Michigan School of Busilness Administration

I I 1 L - ACSI: National Economy, Sectors and Industries American Customer Satisfaction Index'" Manufacturing/ Nondurables 8.2% Apparel Athletic shoes Beer Cigarettes Food processing Gasoline Newspapers Personal care Soft drinks $665 B Manufacturing/ Durables 9.4% Automobiles Consumer electronics Household appliances PC / Printers $582 B Transportation/ Communications/ Utilities 8.8% Airlines Broadcasting/TV Electric service Express delivery Phone/local Phone/longdistance US Postal Service Retail 9.3% Department stores Discount stores Restaurants/ fast food, Supermarkets $392 B Finance/ Insurance 6.8% Banks/ commercial Insurance/life Insurance/ property Services 19.6% Hospitals Hotels Motion pictures $106 B Public Administration/ Government 12.6% I Solid waste (central city, suburb) Police (central city, suburb) IRS I $277 B $62 B $472 B Size of each of 7 sectors shown as % of GDP. Sales figures reflect total sales for the selected U.S. firms. Total sales of selected firms are 51 % of GDP for these 7 sectors and 38% of national GDP. Figure 2 __ -, I

>...~- - ~~~~~~~~. * '^^ is~ ~~~ ''*"' ACSI Model Ovoadll Formal Oeliarbility Overall To Personnel Solis1tac1t i T C om p alson Coifirmfn to Ideal )ilaconflm Expeclttlons c I'ic Tolaranco (lu.volvallon Pelce) Cual tlolllhlllty

I ommmmaw__,, Ai l iu <' aU (' t:;I |(NmI: *.;IIi;(;at (UI IltK'X " The Questions * Customer expectations * Quality * Value * Customer satisfaction * Customer loyalty * Customer complaints Figure 4 I.X4 I- 1

0 I 1L ' AA R National 48 Replicates

ACSI Sample Design Geographically Targeted RDD Sample * Some Companies Only Operate In Selected Geographi Areas - Banks - Utilities - Newspaper Publishers - Hospitals - Some Retail Stores - Etc. * National RDD Sample Is Inefficient And Unnecessary I These Companies Figure 6 C -or 4_

- -. - - - -. -ii -,,:~~ -. - - ~~c —~ u~ - --- ___ ACSI Sample Design Sample Used For Automobile Owners * Only People Who Have Purchased (Not Leased) A New Vehicle Between Six Months and Three Years From Interview Date, And Still Own Vehicle, Are Eligible, Automobile Purchasers Sampled From Dual Frame Design - RDD (As Part of Total ACSI Sample Frame) - List Sample Targeted Toward Brands, Automobile Lists Are An Exception Very Good" Rule - States Require Vehicle Registration To The "Lists Aren't U

- wvarrantv Intnrrmntiirn Ic Dl;+t IARL. r-%. ~* w v %^ -A - a -.- A-,%h - A=%AM&" LA.Z_ M.. —.. ACSI "Short" Questionnaires * One "Short" Questionnaire Maximum Per Respondent * Three Questions * Administered For Products Of Companies Not Measured in ACSI * Only Administered If Respondent is Eligible For Less Than Three Products of Measured Companies > Used For Overall Industry Sector Measurement Figure 8 I mmm r~.r~n

~ zw;- 'i.fiiwt w'. ^iya i -- M -o~i- ii iw~ i1.1. 1 >*ii 11 " ' * ~ ** rr. ~-. - -- I II I I ACSI CATI System — Requirements Respondents Are Eligible To Fill Multiple Quota Cells * Respondents Receive Only a Portion of Possible Screeners Company Quota Cells Can Be Filled Multiple Screeners By Multiple Products From * Multiple Questionnaires * Variable Number of Questionnaires * Write Multiple Data Files From Each Interview - Company Questionnaire Information Written To Separate Data Files - Demographic Information Written To Multiple Data Files

- - ---- - - -- -.4ACl~~~~C ~ u-~~' r II~ ---. -Y.. L -C-IL- -C-~r — L IC~-L~C-L ---- -~~ C~ --- - -~ I~_ ILYLL —~.WY I~~rL~ F-fi ACSI CATI Interview Sequence, Introduction and Respondent Selection - Most Recent Birthday - Screen for 21 or Older if Beer or Cigarette Screeners are Open * Screen for Up to Ten Products - Individual Purchase or Use - Specific Brands Used * Administer Demographic Questionnaire if Eligible for Full Questionnaires * Administer Up To Three Full Questionnaires * Administer Short Questionnaires * Ask Respondent Name for Validation Figure 10 'P

Adz_ = _ -.. _ _w wt_-_+ _*;,, -! Is em — _u-gd i ACSI Interview Design * Product Screening Order - All Consumers Get Screened For Low Use Products Until Quota Filled Within Product Categories, Low Market Share Companies Have Priority Over High Market Share Companies - Consumers Only Get Screened For "Easy to Fill" Products After Failing to Qualify For "Hard to Fill" Companies - Program Allows Some Manipulation To Force Consumers Into "Easy to Fill" Products Before Interview Terminates

..-. - -.,. I.,... - ~ ~ ~ ~ ~ ~ ~ ~ ~ _.._. TRIT r

A (Lr'~~~~ -._..,, — ( -.-..- -- -00 I.-..~- --- - - - __. I F- *IU;kl~jtO:;Ln'SY.'Y -. -Y - _ -. cY:: ~~~~~~~~~~~~~~~~~;~..~. ~. ~ '~,,,Z:. o,.~,~. --- I~~.. Amp- m~~~~~~~~~~~c=~ ~... ACSI Brand Extension Dictionary ACSI Brand Extension and Brand Dictionary - Developed By MSI - Specialized Database Software Built Into CATI System - References "Brands and Their Companies" Database, Purchased From Gale Research, and Other Sources - Updated Annually - Over 4,000 Brand Extensions in Database * Product Dictionary Allows Brands From Multiple Screeners (Product Types) To Be Included in Parent Companv Measl iremFnt

t-Y- -_I - - --- —I -. -- ---- C —1W-.. -— '- YI. shroud I ACSI Brand Extension Dictionary Consumers Tell Interviewer Which Product They Purchased Dictionary Associates That Product With Parent Company - Quota Control by Parent Company - Inserts Product Name in Questionnaire But Inserts Parent Company Name (Code) in Data File "Other - Specify" Products Reviewed To Allow "Nick-Names" To Be Included In Dictionary Figure 14 40I ' ' a

ACSI Brand Extension Dictionary EXAMPLE: Food Processing - Cold Meats, Franks, Poultry Brand Name Armour Eckrich Golden Star Turkey Hebrew National Butcher's Best Poultry Flyers Chicken Holly Farms Bologna Louis Kemp Ball Park Franks Hillshire Sausage Jimmy Dean Sausage Little Sausages Company Name ConAgra ConAgra ConAgra ConAgra Tyson Food Tyson Food Tyson Food Tyson Food Sara Lee Sara Lee Sara Lee Sara Lee Sara Lee'I.

f I I I 4 Di CD (D!-A <Tt

.

M - -.- - no,- '. 1 -mome= w. - h As ibw K'.i(.tl,4 is I ll m':;.~li;.i fl<,4 lhl kx"' 1994 ACSI 74.5 What Does This Mean? Figure 18

Amerrican Custoner Satisfactior Index" ACSI 100 _ 90 70- 60) - 5(.) 50 30 -. 10_ 0'I 0 anacri Manufacturing Services Goverrnment U

II:: I :II II I —: II::: ~:III:11::'1 -I:'III: I ::: 2 - OZ ajn6Tj nr * - t IV I1.1.4

Changes In National ACSI 74.6 74.4 74.2 74.0 73.8 73.6 73.4 73.2 1 — Baseline Q4194 Q1/95 Q2195 Q3/ Figure 21' l, - - I