Mujahid et al. Respond to "Beyond the Metrics for Measuring Neighborhood Effects"

Abstract

In her commentary, Dr. Lynne Messer (1) recognizes the important contributions of our paper (2) to the discussion of methodological issues related to measurement of neighborhood or area-level properties. Dr. Messer reviews the many challenges involved in observational studies of neighborhood health effects, which we and other investigators have noted (3–8). A major challenge is developing theoretical models of the processes through which neighborhoods (or areas) may affect health. Messer argues that our paper "promises more, from a theoretical perspective, than it delivers" (1, p. 869). Our paper is merely a methodological illustration, with no grandiose theoretical aims. However, we do base the measures we explore on a theoretical model of the processes through which residential context may affect cardiovascular disease risk (1, 9). In her discussion of this model, Messer confuses inconsistent empirical support for aspects of the model with the absence of theory itself.

Theorizing on the spatial scale at which different area processes operate is obviously important, but unfortunately there is very little information on which to base this theory. Additional qualitative research on the ways in which individuals interact with spaces may help us develop better theoretical models that may then be empirically tested. However, even if we were able to offer some crude hypotheses regarding spatial scales relevant to different processes, there are features of areas that could plausibly operate at multiple levels. Ultimately, we must rely on empirical research to uncover such relations rather than make a priori assertions under the guise of theory. For this, improving the validity of area-level measures and sensitivity analyses like the ones we present is crucial.

Dr. Messer also alludes to the well-established challenges in estimating causal effects from observational data. Nonexchangeability (or its simpler and less fashionable synonym, "residual confounding") is always a concern. Messer implies that because of this, observational work in neighborhood health-effects research is meaningless. Firm believers in nonexchangeability will accept no defense of observational studies because it is impossible to categorically rule out residual confounding, except in the case of the ideal counterfactual experiment. However, claims of residual confounding also need to be subjected to empirical inquiry: What specific confounders have been omitted, and how strong are their effects expected to be? Careful observational work can empirically examine the sensitivity of results to different degrees of residual confounding and degrees of extrapolation. In this, neighborhood effects research is no different than the rest of epidemiology.

Given the many limitations and logistical challenges of randomized trials (particularly for the study of neighborhood effects), reliance on observational and quasi-experimental data is likely to continue. Hence, anything we can do to improve the rigor of observational work is crucial. Our objective in the current paper was (merely) to contribute to emerging work on the measurement of area-level constructs, not to fully develop a theory on neighborhood causal effects or to resolve the issue of relevant spatial scale. Our objective was not even to estimate associations between neighborhood characteristics and health outcomes. Instead, we wanted to further develop and evaluate our ability to measure area-level constructs.

Epidemiologists are very sophisticated at measuring individual-level characteristics but not as sophisticated at measuring features of ecologic settings. This seriously hampers their ability to examine contextual effects. Our analyses illustrate one approach to quantifying the measurement properties of area-based measures. This approach can be adapted to different constructs and different spatial scales, depending on the research problem and underlying theory. We firmly believe that improving the quality of measurement of area-level constructs is a prerequisite for more rigorous observational work. In fact, several of the inferential problems that arise when area socioeconomic status characteristics are used as proxies for features of areas may be reduced when specific features of areas are examined instead of aggregate socioeconomic status measures (which are, by definition, correlated with individual socioeconomic status, thus magnifying the extrapolation and exchangeability problems). We hope that the illustration we provide in our paper (2) will encourage other investigators to develop and test theoretically relevant area measures and to contrast different approaches to their measurement.

Understanding if and how contexts (including neighborhoods) affect health is challenging and complex, but it is also enormously important from the point of view of public health and policy. In order to answer questions regarding these effects, we need to move beyond blanket (and sometimes facile) critiques, roll up our sleeves, and see if we can improve on the work that has been done to date. This means dealing with a messy, correlated, and confounded reality and doing the best we can to glean truth from our observations. As epidemiologists, this is our job, and also our responsibility to the public.