Approximate spatial layout processing in the visual system: Modeling texture-based segmentation and shape estimation.

Abstract

Moving through the environment, grasping objects, orienting oneself, and countless other tasks all require information about spatial organization. This in turn requires determining where surfaces, objects and other elements of a scene are located and how they are arranged. Humans and other animals can extract spatial organization from vision rapidly and automatically. To better understand this capability, it would be useful to know how the visual system can make an initial estimate of the spatial layout. Without time or opportunity for a more careful analysis, a rough estimate may be all that the system can extract. Nevertheless, rough spatial information may be sufficient for many purposes, even if it is devoid of details that are important for tasks such as object recognition. The human visual system uses many sources of information for estimating layout. Here I focus on one source in particular: visual texture. I present a biologically reasonable, computational model of how the system can exploit patterns of texture for performing two basic tasks in spatial layout processing: locating possible surfaces in the visual input, and estimating their approximate shapes. Separately, these two tasks have been studied extensively, but they have not previously been examined together in the context of a model grounded in neurophysiology and psychophysics. I show that by integrating segmentation and shape estimation, a system can share information between these processes, allowing the processes to constrain and inform each other as well as save on computations. The model developed here begins with the responses of simulated complex cells of the primary visual cortex, and combines a weak membrane/functional minimization approach to segmentation with a shape estimation method based on tracking changes in the average dominant spatial frequencies across a surface. It includes mechanisms for detecting untextured areas and flat areas in an input image. In support of the model, I present a software simulation that can perform texture-based segmentation and shape estimation on images containing multiple, curved, textured surfaces.