Regularized Statistical Methods for Data of Grouped or Dynamic Nature

Abstract

This dissertation consists of two parts. In the first part, one new convex regularized variable selection method is proposed for high-dimensional grouped data. Existing group variable selection methods via convex penalties, such as Yuan and Lin (2006) and Zhao et al. (2009), have the limitation of selecting variables in an ``all-in-all-out'' fashion and lack of selection flexibility within a group. In Chapter II, we propose a new group variable selection method via convex penalties that not only removes unimportant groups effectively, but also keeps the flexibility of selecting variables within an important group. Both the efficient numerical algorithm and high-dimensional theoretical estimation bounds are provided. Simulation results indicate that the proposed method works well in terms of both variable selection and prediction accuracy.

In the second part of the dissertation, we develop the parameter estimation methods for the dynamic ordinary differential equations (ODEs). Ramsay et al. (2007) proposed a popular parameter cascading method that tries to strike a balance between the data and the ODE structure via a ``loss + penalty" framework. In Chapter III, we investigate this method in detail and take an alternative view through variance evaluation on it. We found, through both theoretical evaluation and numerical experiments, that the penalty term in Ramsay et al. (2007) could unnecessarily increase estimation variation. Consequently, we propose a simpler alternative structure for parameter cascading that achieves the minimum variation. We also provide theoretical explanations behind the observed phenomenon and report numerical findings on both simulations and one real dynamic data set.

In Chapter IV, we consider the estimation problem with time-varying ODE parameters. This is often necessary when there are unknown sources of disturbances that lead to deviations from the standard constant-parameter ODE system. To keep the structure of the parameters simple, we propose a novel regularization method for estimating time-varying ODE parameters. Our numerical studies suggest that the proposed approach works better than competing methods. We also provide finite-sample estimation error bounds under certain regularity conditions. The real-data applications of the proposed method lead to satisfactory and meaningful results.