Functional Analytic Perspectives on Nonparametric Density Estimation.

Abstract

Nonparametric density estimation is a classic problem in statistics. In the standard estimation setting, when one has access to iid samples from an unknown distribution, there exist several established and well-studied nonparametric density estimators. Yet there remains interesting alternative settings which are less well-studied. This work considers two such settings. First we consider the case where the data contains some contamination, i.e. a portion of the data is not distributed according to the density we would like to estimate. In this setting one would like an estimator which is robust to the contaminating data. An approach to this was suggested in Kim and Scott (2012). The estimator in that paper was analytically and experimentally shown to be robust, but no consistency result was presented. In Chapter II it is demonstrated that this estimator is indeed consistent for a class of convex losses. Chapter III introduces a new robust kernel density estimator based on scaling and projection in Hilbert space. This estimator is proven to be consistent and will converge to the true density provided certain assumptions on the contaminating distribution. Its efficacy is demonstrated experimentally by applying it to several datasets. Chapter IV considers a different setting which can be thought of as nonparametric mixture modelling. Here one would like to estimate multiple densities with access to groups of samples where each sample in a group is known to be distributed according the same unknown density. Tight identifiability bounds and a highly general algorithm for recovery of the densities are presented for this setting.

Functional analysis is a unifying theme of these problems. Hilbert spaces in particular are used extensively for the construction of estimators and mathematical analysis.