The visual system through the eyes of AI
Using artificial intelligence to understand the visual system in the brain: An international research team (MICrONS) with the participation of the University of Göttingen has developed new AI models to decode the complex processing of visual stimuli in the brain. The researchers investigated how the shape, connectivity pattern and activity of nerve cells in the mouse brain are related. The project's key findings have been published in a series of articles in the journals Nature and Nature Communications.
The image shows more than 1,000 of the 120,000 brain cells (neurons + glia) reconstructed as part of the MICRONS project. Each reconstructed neuron appears in a different, random color. This is a symbolic representation, but not an actual rendering of the dataset – there are far more recorded neurons than those depicted.
Bernstein members involved: Alexander Ecker, Fabian Sinz, Philipp Berens
The study “Foundation Model of Neural Activity Predicts Response to New Stimulus Types and Anatomy” presents a new AI model that has learned from large amounts of data and can be flexibly applied to new tasks. The team analyzed over 135,000 nerve cells in the visual system of mice and developed a model that reliably predicts neuronal responses to new stimuli – even to those it has never seen during training. “Our model can, for example, predict responses to coherent motion patterns, noise patterns and static natural images without ever having been confronted with these types of stimuli,” explains Prof. Dr. Fabian Sinz from the Institute of Computer Science and the Campus Institute of Data Science at the University of Göttingen, who co-developed the model. These types of stimuli are crucial for understanding neural information processing.
In a further study, the team examined the shape and structure of certain nerve cells in the visual area of the brain, known as the visual cortex. “An unsupervised map of excitatory neurons’ dendritic morphology in the mouse visual cortex” shows that the so-called pyramidal cells – cells with a pyramidal shape that transmit important signals to other cells in the visual cortex – are more diverse than previously assumed. The head of the study, Prof. Dr. Alexander Ecker from the same institute, explains: “We have developed machine learning methods that encode the complex 3D shape of a nerve cell in a kind of barcode. These barcodes can then be visualized and analyzed.” Based on 30,000 pyramidal cells, the researchers found that these exhibit fluid transitions between cell types, rather than clearly defined types.
Numerous research institutions were involved in the MICrONS project, which produced the two studies, including Baylor College of Medicine, the Allen Institute for Brain Science, and Princeton University. As part of this project, the team created the “MICrONS Multi-Area Dataset”. It includes both the structure and connectivity of nerve cells and their response characteristics to various visual stimuli. It is currently the largest dataset of its kind ever collected in a mammalian brain. The data were described in the main study “Functional Connectomics Spanning Multiple Areas of Mouse Visual Cortex”.
The models co-developed by the Göttingen researchers were used, among other things, to create a “digital twin” of the nerve cells in the MICrONS data set. This digital twin was able to successfully predict the shape and structure of pyramidal cells without using anatomical information for training. This suggests that the functional and anatomical properties of nerve cells are closely linked.
The research results provide important insights into the organization of the brain and could help to make neuroscientific experiments more efficient in the future. Instead of conducting elaborate and time-consuming experiments in vivo – i.e. in living animals – researchers could first conduct experiments in silico – i.e. in a model – to identify promising hypotheses and then verify them in experiments.
