Skip to main content

ORIGINAL RESEARCH article

Front. Artif. Intell.
Sec. Machine Learning and Artificial Intelligence
Volume 8 - 2025 | doi: 10.3389/frai.2025.1502504

Reconstructing Signal During Brain Stimulation with Stim-BERT: A Self-Supervised Learning Model trained on millions of iEEG files

Provisionally accepted
Karthik Menon Karthik Menon 1Tom Tcheng Tom Tcheng 2Cairn Seale Cairn Seale 2David Greene David Greene 2Martha Morrell Martha Morrell 2,3Sharanya Arcot Desai Sharanya Arcot Desai 2*
  • 1 NeuroPace, Inc., Mountain View, California, United States
  • 2 Biomedical Engineering, NeuroPace, Inc., Mountain View, United States
  • 3 Stanford University, Stanford, California, United States

The final, formatted version of the article will be published soon.

    Brain stimulation has become a widely accepted treatment for neurological disorders such as epilepsy and Parkinson's disease. These devices not only deliver therapeutic stimulation but also record brain activity, offering valuable insights into neural dynamics. However, brain recordings during stimulation are often blanked or contaminated by artifact, posing significant challenges for analyzing the acute effects of stimulation. To address these challenges, we propose a transformer-based model, Stim-BERT, trained on a large intracranial EEG (iEEG) dataset to reconstruct brain activity lost during stimulation blanking. To train the Stim-BERT model, 4,653,720 iEEG channels from 380 RNS System patients were tokenized into 3 (or 4) frequency band bins using 1 second non-overlapping windows resulting in a total vocabulary size of 1,000 (or 10,000). Stim-BERT leverages self-supervised learning with masked tokens, inspired by BERT's success in natural language processing, and shows significant improvements over traditional interpolation methods, especially for longer blanking periods. These findings highlight the potential of transformer models for filling in missing time-series neural data, advancing neural signal processing and our efforts to understand the acute effects of brain stimulation.

    Keywords: selfsupervised learning, machine learning, EEG, Epilepsy, Brain, big data, BERT

    Received: 27 Sep 2024; Accepted: 15 Jan 2025.

    Copyright: © 2025 Menon, Tcheng, Seale, Greene, Morrell and Arcot Desai. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Sharanya Arcot Desai, Biomedical Engineering, NeuroPace, Inc., Mountain View, 30332, United States

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.