Net2Brain: A Toolbox to Compare Artificial Vision Models with Human Brain Responses

Domenic Bersch ^1,2* Martina González Vilas ^1,3 Sari Saba-Sadiya ¹ Timothy Schaumlöffel ^1,2 Kshitij Dwivedi ¹ Christina Sartzetaki ⁴ Radoslaw Martin Cichy ^5,6,7 Gemma Roig ^1,2

• ¹ Department of Computer Science, Goethe University, Frankfurt, Hesse, Germany
• ² The Hessian Center for Artificial Intelligence, Darmstadt, Germany
• ³ Ernst Strüngmann Institute for Neuroscience, Max Planck Society, Frankfurt am Main, Bavaria, Germany
• ⁴ Informatics Institute / VISLab, University of Amsterdam, Amsterdam, Netherlands
• ⁵ Division of Biological Psychology and Cognitive Neuroscience, Department of Education and Psychology, Free University of Berlin, Berlin, Baden-Württemberg, Germany
• ⁶ Berlin School of Mind and Brain, Humboldt University of Berlin, Berlin, Baden-Württemberg, Germany
• ⁷ Bernstein Center for Computational Neuroscience, Humboldt University of Berlin, Berlin, Berlin, Germany

In cognitive neuroscience, the integration of deep neural networks (DNNs) with traditional neuroscientific analyses has significantly advanced our understanding of both biological neural processes and the functioning of DNNs. However, challenges remain in effectively comparing the representational spaces of artificial models and brain data, particularly due to the growing variety of models and the specific demands of neuroimaging research. To address these challenges, we present Net2Brain, a Python-based toolbox that provides an end-to-end pipeline for incorporating DNNs into neuroscience research, encompassing dataset download, a large selection of models, feature extraction, evaluation, and visualization. Net2Brain provides functionalities in four key areas. First, it offers access to over 600 DNNs trained on diverse tasks across multiple modalities, including vision, language, audio, and multimodal data, organized through a carefully structured taxonomy. Second, it provides a streamlined API for downloading and handling popular neuroscience datasets, such as the NSD and THINGS dataset, allowing researchers to easily access corresponding brain data. Third, Net2Brain facilitates a wide range of analysis options, including feature extraction, representational similarity analysis (RSA), and linear encoding, while also supporting advanced techniques like variance partitioning and searchlight analysis. Finally, the toolbox integrates seamlessly with other established open source libraries, enhancing interoperability and promoting collaborative research. By simplifying model selection, data processing, and evaluation, Net2Brain empowers researchers to conduct more robust, flexible, and reproducible investigations of the relationships between artificial and biological neural representations.