On the ordinary ridge regression estimator based on a fixed biasing constant.

Abstract

Since the seminal work of Hoerl and Kennard (1970a), ridge regression has proven to be a useful technique to tackle the multicollinearity problems in the linear regression model, and much research on this topic has been published. A strange phenomenon in this field is that most theoretical results are derived under the original assumption that k is a fixed constant, while most empirical papers seek a value of k as some function of sample data X and y, resulting in the adaptive ridge estimator. However, as Marquadt and Snee (1975) point out, the adaptive ridge estimator loses ground because the original justification of MSE-dominance for a suitably chosen value of k is valid only when k is a fixed constant. Based on this criticism of the adaptive ridge estimator with a stochastic k, this dissertation considers the following issues in relation to the ordinary ridge regression estimator based on a fixed biasing constant: (1) The appropriateness of the matrix mean square error dominance criterion for the justification of the ordinary ridge regression estimator. (2) The choice of a deterministic value of k within the pretesting framework. (3) The statistical analysis of the concavity of the log-likelihood function in the normal linear regression model. Addressing each issue in turn, the dissertation makes three contributions to the field: (1) It shows that the MMSE criterion might not be appropriate for the justification of the ordinary ridge regression estimator, especially when it is assumed that the design matrix is st and ardized so that X$\\sp{\\rm T}$X takes a correlation matrix form. (2) It extends Farebrother's idea (1976) by developing a method for choosing a deterministic value of k. The method tests a newly-identified TMSE-dominance condition for a finite range of k. Numerical examples using actual data illustrate the method. (3) It proves that the log-likelihood function associated with the normal linear regression model is asymptotically concave with probability one. The practical implication of this finding is discussed and a table of this probability is constructed for various sample sizes.