Efficient algorithms for estimating the width of nearly normal distributions

Abstract

Typical physics data samples often conform to Gaussian distributions with admixtures of more slowly varying backgrounds. Under such circumstances the standard deviation is known to be a poor statistical measure of distribution width. As an alternative, the performance of Gini's mean difference is compared with the standard deviation and the mean deviation. Variants which sum over subsets of all possible pairs are shown to have statistical efficiencies comparable to the mean difference and mean deviation but do not require extensive data storage or a priori knowledge of the sample mean. These statistics are reasonable candidates for monitoring the distribution width of a real time data stream.