Logistic-Normal Mixtures with Heterogeneous Components and High Dimensional Covariates.

Abstract

Finite mixture regression (FMR) models are powerful modeling tools to analyze data of various types because of FMR's flexible model structure and appealing interpretation. Applications can be found in a variety of areas, such as economics, finance and clinical trails. In this dissertation, we focus on the logistic-normal mixture models. The key difference between the logistic-normal mixtures and most other FMR models is that both the component means and the mixing parameters in the logistic-normal mixtures could depend on covariates. This unique feature makes the model useful in both applications and interpretations but also renders the theoretical development and data analysis more difficult.

We show the consistency of the parameter estimation based on a penalized maximum likelihood method, when the sample size increases but the number of the covariates is fixed. We then consider the case where the number of the potential covariates grows with the sample size. In this setting, because of the non-convex log-likelihood function, the traditional techniques for analyzing model selection for high dimensional data do not work. For known number of components, we utilize the empirical process theory and show under appropriate conditions that the LASSO type estimators remain consistent in terms of Kullback-Leibler divergence. When the number of components is unknown, we develop a new selection criterion (SCMM) for mixture models that can estimate the number of components consistently. We demonstrate our theory both in simulated data and real data.