AUTHOR=Wang Yuwei , Zeng Yi 

TITLE=Multisensory Concept Learning Framework Based on Spiking Neural Networks

JOURNAL=Frontiers in Systems Neuroscience

VOLUME=16

YEAR=2022

URL=https://www.frontiersin.org/journals/systems-neuroscience/articles/10.3389/fnsys.2022.845177

DOI=10.3389/fnsys.2022.845177

ISSN=1662-5137

ABSTRACT=<p>Concept learning highly depends on multisensory integration. In this study, we propose a multisensory concept learning framework based on brain-inspired spiking neural networks to create integrated vectors relying on the concept's perceptual strength of auditory, gustatory, haptic, olfactory, and visual. With different assumptions, two paradigms: Independent Merge (IM) and Associate Merge (AM) are designed in the framework. For testing, we employed eight distinct neural models and three multisensory representation datasets. The experiments show that integrated vectors are closer to human beings than the non-integrated ones. Furthermore, we systematically analyze the similarities and differences between IM and AM paradigms and validate the generality of our framework.</p>