Breaking down difficult problems into simpler parts and, conversely, composing primitive units to generate rich behavior, are hallmarks of biological intelligence. Principles of reductionism and compositionality may also guide how we process the complex visual environment of our everyday experience—that is, natural visual processing. This talk will be centered on a core element of this strategy, broadly termed segmentation and integration, by which our visual system organizes complex visual inputs into groups corresponding to distinct perceptual objects. I will present progress made over the years by my lab and collaborators through a tight integration of theory, computational modeling, visual neurophysiology and psychophysics.

First, I will present a normative framework that explains widely observed properties of neural coding in primary visual cortex (V1). Second, building on that foundation, I will argue that a complete understanding of V1 phenomenology must account for the non-stationary statistics of natural inputs. The theory makes detailed predictions about the sensitivity of V1 neurons to cues for integration and segmentation, including a surprising flexibility of functional interactions that we have confirmed recently with data recorded from V1 populations in macaques. The third piece will focus on perceptual segmentation and integration in human observers. Extending our computational framework to deep probabilistic algorithms for natural image segmentation, leads to a surprising prediction about the time course of perceptual segmentation. I will provide empirical evidence recorded in a new experimental paradigm we have developed to measure perceptual grouping of natural stimuli with human participants. This effect challenges influential theories of the time-course of perceptual organization. Our model explains it as a signature of iterative Bayesian inference, offering a normative foundation for recent semi-mechanistic models based on artificial RNNs and a path to better align them with human perception.