Biased and unbiased Cramer-Rao bounds: Computational issues and applications.

Abstract

In most parametric estimation problems there exists a trade-off between bias and variance of the estimator. The performance of different estimators can be effectively compared by putting them on a bias-variance trade-off plane. Recently Hero presented an estimator independent 'uniform' Cramer-Rao (CR) bound on estimator variance by separating the bias-variance trade-off plane into achievable and unachievable regions. However, the application of the uniform CR bound is restricted since it involves inversion of a Fisher information matrix (FIM). The inversion can become computationally intractable when the number of parameters to be estimated is large. We present several recursive algorithms to compute columns of the inverse of the FIM that can result in significant computational savings. These algorithms are based on a monotonically convergent geometric series (GS) decomposition of the inverse, and a non-monotonically convergent conjugate (CG) gradient algorithm. We also present two recursive algorithms, based on the GS and the CG, to efficiently compute the columns of the pseudo-inverse of the FIM, which may be used to compute the uniform CR bound for the rank-deficient problems. In order to place an estimator on the bias-variance trade-off plane for comparison to the bound, the estimator variance, the estimator bias and bias gradient must be evaluated. We present a simple and accurate method for experimentally determining the variance, bias and bias gradient based on sample averaging the score function. The original formulation of the uniform CR bound only applies to full-rank problems resulting in a non-singular FIM. We extend the uniform bound to allow the singular FIM. We apply the methods developed in this thesis to several different examples including two-dimensional SPECT image reconstruction, one-dimensional discrete deconvolution, and one-dimensional edge localization.