October 25, 2024 by Ingrid Fadelli , Tech Xplore
Collected at: https://techxplore.com/news/2024-10-ai-mimics-neocortex-winner-approach.html
Over the past decade or so, computer scientists have developed increasingly advanced computational techniques that can tackle real-world tasks with human-comparable accuracy. While many of these artificial intelligence (AI) models have achieved remarkable results, they often do not precisely replicate the computations performed by the human brain.
Researchers at Tibbling Technologies, Broad Institute at Harvard Medical School, The Australian National University and other institutes recently tried to use AI to mimic a specific type of computation performed by circuits in the neocortex, known as “winner-take-all” computations.
Their paper, published on the bioRxiv preprint server, reports the successful emulation of this computation and shows that adding it to transformer-based models could significantly improve their performance on image classification tasks.
“Our recent paper was inspired by the incredible computational capabilities of the mammalian brain, particularly the neocortex,” Asim Iqbal, first author of the paper, told Tech Xplore.
“Our primary objective was to take inspiration from how the brain processes information and apply those principles to improve artificial intelligence systems. Specifically, we focused on a computation called ‘winner-take-all’ that seems to be a fundamental operation in cortical circuits.”
“Winner-take-all” is a biological mechanism that occurs when one or a few neurons within a set (i.e., the one/ones with the highest activation level) influence the outcome of a computation. The more active neurons essentially suppress the activity of other neurons, becoming the only cells contributing to a specific decision or computation.
Iqbal and his colleagues tried to realistically mimic this biological computation using neuromorphic hardware and then use it to improve the performance of well-established machine learning models. To do this, they used IBM’s TrueNorth neuromorphic hardware chip, which is especially designed to mimic the brain’s organization.
“Our biophysical network model aims to capture the key features of neocortical circuits, focusing on the interactions between excitatory neurons and four major types of inhibitory neurons,” explained Iqbal.
“The model incorporates experimentally measured properties of these neurons and their connections in the visual cortex. Its key feature is the ability to implement ‘soft winner-take-all’ computations, where the strongest inputs are amplified while weaker ones are suppressed.”
By performing these brain-inspired computations, the team’s approach can enhance important signals, while filtering out noise. The key advantage of their NeuroAI system is that it introduces a new biologically-grounded and yet computationally efficient approach to processing visual information, which could help to improve the performance of AI models.
“One of our most exciting achievements was the successful implementation of our brain-inspired computations on IBM’s TrueNorth neuromorphic chip,” said Iqbal.
“This demonstrates that we can translate principles from neuroscience to real hardware. We were also thrilled to see significant improvements in the performance of Vision Transformers and other deep learning models when we incorporated our winner-take-all inspired processing. For example, the models became much better at generalizing to new types of data they hadn’t been trained on—a key challenge in AI.”
Iqbal and his colleagues combined the soft winner takes all computations performed using their approach with a vision transformer-based model. They found that their approach significantly improved the model’s performance on a digital classification task for completely “unseen” data through zero-shot learning.
In the future, their brain-inspired computing approach could be applied to other AI systems for a wide range of applications, including computer vision, medical image analysis and autonomous vehicles. Meanwhile, the researchers plan to investigate how the same brain-inspired principles underpinning their approach could be leveraged to tackle more complex cognitive tasks.
“We’re particularly interested in implementing working memory and decision-making processes using our approach,” added Iqbal.
“We also plan to investigate how we can incorporate learning mechanisms inspired by the brain, which could lead to AI systems that can learn and adapt more efficiently. Additionally, we’re keen to test our approach on other emerging neuromorphic hardware platforms to further bridge the gap between neuroscience and AI.”
More information: Asim Iqbal et al, Biologically Realistic Computational Primitives of Neocortex Implemented on Neuromorphic Hardware Improve Vision Transformer Performance, bioRxiv (2024). DOI: 10.1101/2024.10.06.616839
Journal information: bioRxiv
Leave a Reply