How the brain detects relevance — and how AI learns from it
The human brain’s ability to filter relevant information from the vast amount of data it continuously receives is known as attention. Researchers at BIFOLD at the Technical University of Berlin, in close collaboration with scientists from the Kording Lab at the University of Pennsylvania, have developed a brain-inspired AI model of visual attention. What is particularly remarkable is that the model reproduces numerous well-established behavioral and perceptual phenomena from psychology and neurophysiology without these effects having been explicitly programmed into it. This not only advances our understanding of human visual perception but also provides a new perspective and framework for AI researchers. The joint study has now been published in the journal Nature Communications.
Our sensory organs provide the brain with far more information than it can process. It must therefore filter out the relevant information from this flood of data. This process is called attention. © Unsplash/Google DeepMind
Bernstein member involved: Klaus-Robert Müller
The researchers introduce a comprehensive neuroscience-inspired approach that unifies several forms of attention – spatial, feature-based, and object-based – within a single model. In doing so, the work addresses one of the central unresolved problems in cognitive neuroscience, the so-called “binding problem.” Human sensory systems continuously take in far more information than the brain can process. Yet people can effortlessly focus on linking the most important information: a red traffic light while driving, a child at the side of the road, or a conversation in a very noisy room. This ability is known as attention. It allows the brain to select relevant information from a flood of inputs or to bind different visual features into coherent objects and guide behavior. How the brain processes relevant information, however, has long remained a mystery in neuroscience.
At the core of the publication is a novel model architecture for visual processing. The model combines a so-called feedforward pathway for feature extraction with a top-down, attention-driven pathway that is modulated by bidirectional recurrent control mechanisms. This enables the model to learn to solve multiple visual tasks simultaneously. It is designed to support bottom-up, top-down, recurrent, and lateral information processing and communication, and to process both low-level stimuli (e.g., images) and high-level inputs (e.g., tasks and cues) in parallel.
“Our goal is to make neural networks more brain-like”
“The resulting model is able to successfully perform the characteristic behaviors of human attention: orienting, filtering, and visual search in complex visual scenes,” explains Saeed Salehi, scientist in the Machine Learning group at BIFOLD chaired by Prof. Dr. Klaus-Robert Müller. An artificial system was created that not only masters recognition and segmentation tasks, but also demonstrates classic attentional functions: orienting toward relevant stimuli, filtering out distracting information, and goal-directed search. In extensive experimental series, the team shows that the model not only improves object recognition in complex scenes, but also replicates psychophysical effects such as perceptual load and inattentional blindness. At the same time, its internal units develop neuronal properties consistent with primate physiology, including multiplicative modulation of neuronal activity when attention shifts and the encoding of boundary information. “Attention, object perception, and the binding of stimuli are among the brain’s most important capabilities. Translating these into practical AI architectures holds great potential,” says Klaus-Robert Müller.
The practical implications of this work could be far-reaching for both neuroscience and AI research. By linking insights from psychology, neuroscience, and computational modeling, the study advances our understanding of how attention and object perception function in the brain. It represents a step toward better models of human cognition and opens up applications ranging from the diagnosis of visual attention disorders to the development of novel adaptive AI architectures.
Konrad Kording, Penn Integrated Knowledge Professor, University of Pennsylvania as well as Co-Director CIFAR Learning in Machines & Brains Program: “Neural networks generally process information in only one direction, whereas the brain is bidirectional. Neural networks have a single task: extracting information. The brain, by contrast, performs a wide range of tasks, both bottom-up and top-down. Our goal is to make neural networks more brain-like and, in doing so, to develop new models of the brain and of AI.”
