Some New Results on Ridge Regression Estimation

Abstract

In this paper we consider various interpretations of the ordinary ridge regression estimator with a given shrinkage factor k, and report the results of an extensive Monte Carlo of several ridge regression estimators involving sample-based rules for selecting k. A major distinguishing feature of the study is the use of a general loss structure, the p-norm, in the evaluation process. Other factors taken into consideration include different degree of ill-conditioning of data, different number of explanatory variables, and different shape and non-centrality of the regression coefficients. The main results are: (i) With minor exceptions, all the ridge regression estimators considered yield a smaller average loss regardless of the loss function used. (ii) The reduction in the average loss of the ridge regression estimators increases when the degree ill-conditioning of data increases. The reduction reaches a substantial level when the degree of ill-conditioning is only moderate. (iii) On the basis of our experiment it is possible to make a recomendation concerning the rule of k.