Language of Administration as a Source of Measurement Error: Implications for Surveys of Immigrants and Cross-cultural Survey Research.

Abstract

U.S. surveys of minorities and immigrants allow respondents to answer in the language of their choice; however, the extent to which the language of survey administration may affect survey responding remains unknown. Psychological and psycholinguistic research demonstrates that the language of administration may influence responses through its impact on various cognitive processes and through the cultural frame that a language brings to the conversational context. This dissertation examines the extent to which language of administration influences bilingual bicultural respondents’ answers to survey questions. It consists of three related essays. The first proposes a framework for possible language influences at each step of the response formation process, based on findings from psycholinguistics and cross-cultural psychology. The second examines the existence of language effects in observational data from the New Immigrant Survey. It compares groups of questions where language effects are expected based on divergent view points of the two cultures under study (Hispanic vs. American) and groups of questions where no such effects were expected. The absence of random assignment to language of interview is addressed through the use of a propensity score method. The last essay presents analyses of an experimental assignment of bilingual respondents to language of survey administration in the National Latino and Asian American Study, comparing questions on similar topics as the ones examined in the observational study. Overall, the empirical investigation fails to detect strong language influences. Consistent with the proposed hypotheses, significant differences between language groups are found in questions related to alcohol consumption and family pride. Surprisingly, no language effects are detected in questions related to physical and mental health, family cohesion and family conflict. Various factors may explain the lack of strong language effects in this investigation – acculturation and level of bilingualism of the sample, interviewer and survey design characteristics, and unobservable factors not accounted in the propensity models are the most obvious. The existence of some language effects, however, requires further investigation of the conditions and mechanisms that produce them.