Enabling Responsible Data Science through Multi-Dimensional Data Management

Abstract

In today’s world, data is collected and utilized at an unprecedented scale, profoundly influencing society. Big data enables analyses that guide high stakes decisions and supports data-driven systems. While the benefits of big data are significant, the challenges extend beyond efficient processing and storage; as data scientists, we are responsible for ensuring that data applications ethically benefit society.

Data science technologies can cause harm if they reinforce inequities, particularly when sensitive data—such as data linked to protected characteristics like race and gender is mishandled. This dissertation contributes to responsible data science by proposing comprehensive data management techniques to address challenges throughout the big data lifecycle. These approaches aim to enhance fairness, transparency, and accountability in data systems while considering the complexities associated with multiple protected characteristics.

Firstly, data acquisition often results in the underrepresentation of certain populations, which risks perpetuating unfair treatment and oversight of these groups. Obtaining representative samples becomes particularly challenging when dealing with intersectional subgroups. We propose coverage analysis techniques to efficiently identify representation bias in multi-table databases, guiding data users toward obtaining more representative samples. Secondly, biases embedded in historical decisions can propagate into downstream machine learning tasks, resulting in unfair predictions. We emphasize the importance of addressing the root causes of unfairness in the training data. We propose model agnostic data pre-processing techniques to effectively detect and mitigate biased data collection, thereby enhancing ma- chine learning fairness across subgroups. Thirdly, data analytics based on cherry-pick generalizations can lead to misleading insights, diminishing the experiences of certain subgroups in decision-making. We refine these generalizations across multiple attributes to develop a framework evaluating their appropriateness, identify subgroup discrepancies, and promote more accurate and inclusive representations of data. Lastly, when a data analysis pipeline produces unexpected outputs, it is the responsibility of data scientists to interpret the potential sources of error. We propose a row-level data lineage approach to enhance pipeline transparency, enabling them to trace the origins of issues.