Personalized and Distributed Data Analytics in Heterogeneous Environments

Abstract

It is a common wisdom in statistics that more data leads to better models. However, as data are increasingly collected from distributed sources, such as different devices or users, their inherent statistical heterogeneity creates challenges for effective knowledge integration. Conventional population-based models often rely on i.i.d. assumptions, which often neglect variations across data sources. When data distributions differ, understanding their structure and integrating information for predictive modeling becomes non-trivial.

This dissertation tackles these challenges through personalized modeling. Instead of fitting one single model for data from all sources, personalized data analytics fits data source-specific models while still encouraging knowledge transfer across sources.

This dissertation proposes personalized descriptive and predictive analytics that attempt to answer three key questions: (Q1) How can we develop descriptive analytics to extract shared and unique patterns from heterogeneous data? (Q2) How can we design robust statistical methods that remain reliable in the presence of outliers? (Q3) How can we leverage insights from covariate and concept shifts to construct effective personalized predictive models?

To answer these questions, the dissertation proposes three methodological contributions. Chapter 2 proposes Personalized PCA (PerPCA), a novel approach that distinguishes shared and unique features across data sources using mutually orthogonal global and local principal components. Chapter 3 presents Triple Component Matrix Factorization (TCMF) to recover global, local, and noisy components in multi-source data corrupted by outlier noise. Both PerPCA and TCMF are equipped with theoretical guarantees on statistical errors. Chapter 4 develops a predictive modeling framework called Personalized Federated Learning via Domain Adaptation (PFL-DA) that addresses both covariate and concept shifts across distributed sources.

The proposed methods provide scalable and interpretable solutions for extracting insights, integrating knowledge, and improving predictive performance in distributed and heterogeneous environments. These findings have broad applications across various domains, including image and video processing, topic modeling, and manufacturing.