Klaus Gramann1*
Fabien Lotte2
Frederic Dehais3
Hasan Ayaz4
Mathias Vukelić5
Waldemar Karwowski6
Stephen Fairclough7
Anne-Marie Brouwer8
Raphaëlle N. Roy3- 1Biological Psychology and Neuroergonomics, Technische Universitaet Berlin, Berlin, Germany
- 2Inria Center at the University of Bordeaux/LaBRI, Bordeaux, France
- 3Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, Toulouse, France
- 4School of Biomedical Engineering Science and Health Systems, Drexel University, Philadelphia, PA, United States
- 5Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, Germany
- 6Department of Industrial Engineering and Management Systems, University of Central Florida, Orlando, FL, United States
- 7School of Psychology, Liverpool John Moores University, Liverpool, United Kingdom
- 8TNO Soesterberg, Radboud University Nijmegen, Nijmegen, Netherlands
Editorial on the Research Topic
Open science to support replicability in neuroergonomic research
Open access to data, code, and protocols in research is vital for accelerating scientific progress, ensuring transparency and reproducibility, and maximizing resource utilization. By making datasets and code freely available, researchers can collaborate across disciplines, verify, replicate, extend findings, and innovate more efficiently. Additionally, open data and collective use such as in open competitions (Roy et al., 2022) support education, promote equity in research opportunities, and enhance public trust and engagement. These benefits collectively advance the application of scientific principles, fostering a more inclusive and robust scientific community. However, within the neuroergonomics and neurotechnology community, this process has been remarkably slow for several reasons, resulting in a minimal number of freely available data sets with often reduced data information (Hinss et al., 2021).
Replicability and reproducibility are crucial, especially in neuroergonomics, which focuses on understanding neural processes in realistic settings that often involve resource-intensive and complex neurophysiological data collection methods like electroencephalography (EEG) or functional near-infrared spectroscopy (fNIRS). These experiments require costly and time-consuming setups, such as simulators or real vehicles. Additionally, inter- and intra-subject variabilities limit the serviceability of datasets. This Research Topic (RT) thus invited submissions that support data sharing of such complex and expensive (neuro-)physiological data collected in naturalistic or real-life conditions, and that investigate different aspects of the user interacting with technical systems. Besides sharing data sets, authors were encouraged to provide their signal processing code using open-source software/platforms and to follow the guidelines from the Organization for Human Brain Mapping (OHBM) and the Brain Imaging Data Structure (BIDS) (Pernet et al., 2019) for data labeling and organization.
This RT accepted four submissions providing valuable multimodal data that can be used by the community to replicate and extend the reported analyses. The first data set contributed by Kueper et al. provides 64-channel EEG and 8-channel EMG data of 8 healthy participants interacting with an exoskeleton. The original study in which the data was collected investigated whether tactile detection of incorrect behavior in an exoskeleton or orthosis evokes similar event-related potentials (ERPs) as visually observed incorrect behavior to use the resultant information to correct the behavior. To this end, the authors introduced easily recognizable force direction errors, opposing the planned body movements, and recorded behavioral, EEG, and EMG data during normal and erroneous interactions with the exoskeleton. It should be noted that this data set was used for an IJCAI 2023 competition entitled: “IntEr-HRI Competition: Intrinsic Error Evaluation during Human-Robot Interaction,” hence fostering the ideas of several research teams to further the development of passive BCIs and adaptive Human-Robot Interaction (HRI).
The second contribution, by Pan et al., provides an extensive data analysis pipeline based on the data published by Kueper et al.. The pipeline received a prize as the winning approach of the above-mentioned competition. The authors focus on developing an automated pipeline for online error detection in an assistive device for improving the reliability of orthosis movements in real-time. This approach makes two major contributions to the field, the first demonstrating the feasibility of integrating two temporal derivative features with an effect size-based feature selection strategy, specifically for online EEG-based BCIs. The second contribution is the introduction of an innovative approach for continuous online error prediction, incorporating a simple noise rejection technique to reduce false alarms. The methodology allows for continuous error detection, with the potential to significantly advance practical applications in neuroadaptive technology and HRI.
Gehrke et al. contribute with a dataset that addresses one of the key challenges in designing immersive virtual reality (VR) experiences: creating an environment that closely mimics the real world to enhance users' sense of presence. The available Mobile Brain/Body Imaging data (Gramann et al., 2011; Jungnickel et al., 2018) in Motion-BIDS format (Jeung et al., 2024) includes brain recordings via 64-channel EEG together with behavioral responses, and motion capture during an interactive VR experience in which participants were asked to point to a virtual object that appeared at different locations in front of them (see Gehrke et al., 2019, 2022). In three different sensory conditions, only visual, visual plus tactile, or visual plus tactile combined with force feedback are provided. When touching the virtual object sensory feedback about the successful touch would be provided (visual only; visual plus tactile; visual, tactile, and force feedback) indicating the completion of the trial. In some trials, the feedback was provided prematurely, creating an error in the interaction. By violating the users' predictions about the VR's interaction, prediction errors were recorded and analyzed providing a multimodal prediction error data set that allows for investigating user presence in the virtual scenario.
Finally, Demirezen et al.'s systematic literature review summarizes the current efforts toward reproducibility in both machine learning and EEG research related to mental workload. They formulate guidelines structured around the Cross-Industry Standard Process for Data Mining (CRISP-DM) framework, aiming to ensure methodological transparency and comprehensiveness while enhancing collaboration, knowledge-sharing, and the reliability of EEG and machine learning techniques. The authors further extracted machine learning studies using EEG to estimate mental workload and assessed the reproducibility of these studies using the previously described guidelines, highlighting both studied and overlooked areas and identifying current challenges. Consistent with the topic of the RT, one of the identified challenges was the lack of open sharing of data and code.
The limited number of published data sets, open analysis pipelines, and guidelines for reproducible research in neuroergonomics are highlighted in this Research Topic marking an essential initial step toward enhancing open-science practices in the field. This RT aims to encourage the sharing of complex and costly neurophysiological data collected under naturalistic conditions, and robust analysis pipelines, in order to facilitate replication, validation, and extension of studies. These efforts not only foster a collaborative and transparent scientific community but also pave the way for faster translation of neuroergonomics toward practical applications. As more researchers adopt these open-science approaches, the potential for groundbreaking advancements and everyday usability in neuroergonomics will significantly increase, ultimately driving the field toward more inclusive and impactful scientific progress.
Author contributions
KG: Conceptualization, Writing – original draft, Writing – review & editing. FL: Conceptualization, Writing – review & editing. FD: Conceptualization, Writing – review & editing. HA: Conceptualization, Writing – review & editing. MV: Conceptualization, Writing – review & editing. WK: Conceptualization, Writing – review & editing. SF: Conceptualization, Writing – review & editing. A-MB: Conceptualization, Writing – review & editing. RR: Conceptualization, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Gehrke, L., Akman, S., Lopes, P., Chen, A., Singh, A. K., Chen, H.-T., et al. (2019). “Detecting visuo-haptic mismatches in virtual reality using the prediction error negativity of event-related brain potentials,” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, 1–11.
Gehrke, L., Lopes, P., Klug, M., Akman, S., and Gramann, K. (2022). Neural sources of prediction errors detect unrealistic VR interactions. J. Neural Eng. 19:e036002. doi: 10.1088/1741-2552/ac69bc
Gramann, K., Gwin, J. T., Ferris, D. P., Oie, K., Jung, T. P., Lin, C. T., et al. (2011). Cognition in action: Imaging brain/body dynamics in mobile humans. Rev. Neurosci. 22:47. doi: 10.1515/RNS.2011.047
Hinss, M. F., Somon, B., Dehais, F., and Roy, R. N. (2021). “Open EEG datasets for passive brain-computer interface applications: lacks and perspectives,” in 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER), Italy, 2021, 686–689.
Jeung, S., Cockx, H., Appelhoff, S., Berg, T., Gramann, K., Grothkopp, S., et al. (2024). Motion-BIDS: an extension to the brain imaging data structure to organize motion data for reproducible research. Sci. Data 11:716. doi: 10.1038/s41597-024-03559-8
Jungnickel, E., Gehrke, L., Klug, M., and Gramann, K. (2018). “MoBI–mobile brain/body imaging,” in Neuroergonomics (Cambridge: Academic Press), 59–63.
Pernet, C. R., Appelhoff, S., Gorgolewski, K. J., Flandin, G., Phillips, C., Delorme, A., et al. (2019). EEG-BIDS, an extension to the brain imaging data structure for electroencephalography. Sci. Data 6:103. doi: 10.1038/s41597-019-0104-8
Keywords: open access, open data, open analysis, replicability, EEG, neuroergonomics
Citation: Gramann K, Lotte F, Dehais F, Ayaz H, Vukelić M, Karwowski W, Fairclough S, Brouwer A-M and Roy RN (2024) Editorial: Open science to support replicability in neuroergonomic research. Front. Neuroergon. 5:1459204. doi: 10.3389/fnrgo.2024.1459204
Received: 03 July 2024; Accepted: 08 July 2024;
Published: 30 July 2024.
Edited and reviewed by: Felix Putze, University of Bremen, Germany
Copyright © 2024 Gramann, Lotte, Dehais, Ayaz, Vukelić, Karwowski, Fairclough, Brouwer and Roy. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Klaus Gramann, a2xhdXMuZ3JhbWFubiYjeDAwMDQwO3R1LWJlcmxpbi5kZQ==