Data Dilemmas: The Science and Politics of Communicating Uncertainty in Human Rights Information

Abstract

Data and statistics about crime and human rights violations are incomplete and biased, yet numbers are in high demand. Advocates and policymakers often tally up available, yet partial data and present them as hard numbers to bring attention to abuses and to influence aid and accountability. As calls for transparency about data limitations increase, I ask two related questions: 1) How do human rights advocates think about the value of quantification and its associated uncertainty when using it to inform and influence audiences? 2) With respect to quantitative evidence about violence, crime or abuse, how do different presentations of data uncertainty affect decision outcomes? Using mixed methods – qualitative and experimental – this research teases out the political, behavioral and methodological challenges that advocates face as they collect, communicate, and deploy violence statistics in global and local human rights advocacy contexts. Semi-structured interviews with twenty-eight frontline human rights advocates (focused on global, U.S., or Colombian issues) reveal that data uncertainty is an unavoidable reality in human rights work, and advocates are keenly aware of this. Advocates mostly share consistent and as-of-yet unrecognized ideas and practices about what could be called “good enough numbers” for advocacy. Central to these practices are pragmatic, yet principled tradeoffs that pull advocates away from strictly rigorous treatment of data and uncertainty. Transparency is a key issue that advocates somewhat reluctantly reduce in pragmatic considerations of benefits and risks.

The survey experiment employed three vignettes and four uncertainty messages, designed on the basis of science communication theory and human rights communication practices, to explore the impact of “being transparent” about data limitations. Responses from 970 college graduates confirm that 1) numbers have strong anchoring effects, but also show that 2) simple caveats about uncertainty do little to de-anchor decision-making. The research also finds novel evidence that 3) only one message type – called here “expert interception” – effectively drives people to account for uncertainty in their decisions (replicating earlier findings about communicating uncertainty in weather forecasting (Joslyn & LeClerc, 2013)). Finally, 4) while different studies suggest perceptions of trustworthiness of information providers may increase, or decrease, with different levels of uncertainty information, this study finds minimal fluctuation in source trust across any of the tested uncertainty messages.

Information providers face a clear choice in allowing numbers to speak for themselves or proactively mitigating bias through language – a choice that is inherently political. It appears that uncertainty is most effectively conveyed when communicators intercept the power of numbers to project “mechanical objectivity” with their expert knowledge about the data generation process and data limitations. A core theoretical contribution of this dissertation is the elucidation of a “rigor-pragmatism continuum” – a novel framework informed by the research findings. The continuum challenges the good-bad dichotomy that is common in critiques of human rights numbers and offers an alternative to support more nuanced analysis about how human rights advocates wrestle with using uncertain numbers. As a whole, this dissertation has wide-reaching implications for human rights and science communication scholarship and practice.