Building an open-source community to enhance autonomic nervous system signal analysis: DBDP-autonomic

Smartphones and wearable sensors offer an unprecedented ability to collect peripheral psychophysiological signals across diverse timescales, settings, populations, and modalities. However, open-source software development has yet to keep pace with rapid advancements in hardware technology and availability, creating an analytical barrier that limits the scientific usefulness of acquired data. We propose a community-driven, open-source peripheral psychophysiological signal pre-processing and analysis software framework that could advance biobehavioral health by enabling more robust, transparent, and reproducible inferences involving autonomic nervous system data.

1 Introduction

Chronic physical conditions and mental health disorders are increasingly prevalent (1). Integrating biological, behavioral, and environmental data is needed to enable early detection, just-in-time intervention, and outcome monitoring to promote biobehavioral health.

Mobile monitoring technologies are changing how we collect behavioral, environmental, and peripheral psychophysiological data in non-clinical settings. They allow researchers and clinicians to gather information continuously and unobtrusively beyond traditional clinical settings’ temporal and spatial limitations. While there have been successful examples of technology-driven biobehavioral applications such as detecting stress by continuously monitoring heart rate variability (2, 3), predicting episodes of depression (4, 5), or identifying sleep disorder patterns using wearable sleep trackers (6), the full potential of these technologies has yet to be realized. There are at least two key challenges hindering progress: (1) the common misinterpretation of peripheral psychophysiological signals in biobehavioral research and (2) the need for greater transparency and reproducibility in biobehavioral research that involves peripheral physiological data.

1.1 The criticality of context in the interpretation of autonomic nervous system data

The Autonomic Nervous System (ANS) plays a fundamental role in regulating myriad physiological processes within the body, including, but not limited to, heart rate, blood pressure, respiration, and digestion. Its primary function is maintaining homeostasis by continuously adjusting the body to changing internal and external conditions. Thus, understanding the context in which ANS data is collected is critical to making accurate and meaningful biobehavioral interpretations.

The ANS is highly variable within and across people but also has stable characteristics based on an individual’s baseline biology. ANS activity can be influenced by various factors, including but not limited to, stress, affect, cognition, physical activity, sleep, illness, medications, and environmental demands. ANS activity also varies over time—for example, heart rate variability changes during different stages of sleep (7). Further, ANS responses differ between individuals based on age, genetics, and health conditions (8, 9). The combination of these influencing factors makes the ANS particularly difficult to study, especially when the broader context (setting, activity, health status, etc.) and a person’s baseline state are not considered.

Fluctuations in ANS signals may be misattributed to one factor (e.g., a stressful event) when they are the result of another factor (e.g., physical activity) (Figure 1). Some common examples of transient states that are misinterpreted when context is excluded include affect (emotion/mood (10)), cognition (challenge/threat (11)), and physical perturbations (sleep, medications, exercise (12)). This impacts broader digital biomarker development that focuses on a single physiological system and thus ignores the broader context and systemic interconnectedness that may collectively influence diagnostic outcomes (autonomic neuropathy, neurodegenerative disease, gastrointestinal disorders, etc.). Context also helps account for psychophysiological differences by differentiating between a stimulus-driven or condition-specific shift in ANS activity vs. natural fluctuations (e.g., diurnal variations) around a baseline. Additionally, including context in research and clinical care enables a more quantitative assessment of the effects of interventions like medications and therapies and whether observed changes constitute lasting outcomes, which are notoriously difficult to assess systematically.

Figure 1

Figure 1. This figure demonstrates the need for additional context when analyzing ambulatory physiological signals. We used a stress-prediction model on heart rate variability data to predict a probability of physiological stress/arousal for a person over 24 h (a). From a naïve interpretation, it seems there are three major stressful (or high arousal) periods (b). However, when asked to self-report stress levels, the user rated a mix of low to high stress for those periods, contradictory to the purely physiological interpretation (c). Considering additional contextual information, we realize that only one high-arousal episode was stressful since the user was undergoing an exam. The other periods were when the user was in a class and exercising later during the day, which showed similar physiological arousal but were not stressful (d).

In sum, it is important to consider internal, external, and temporal contexts to ensure accurate diagnosis and interpretation of ANS using digital health data. Software that enables data fusion and multi-modal analysis is needed to address these issues adequately. This approach differs from existing data analytic pipelines that explore one sensing channel at a time. Integrating data types could fill the gap in digital health initiatives that include ANS data and thus address one of the major issues in this area.

1.2 Addressing reproducibility

Scientific endeavors are experiencing both a crisis of method reproducibility (the ability to achieve congruent results from a given dataset and analysis) and results reproducibility (the ability to recreate results independently (13, 14)). Large-scale projects indicate various degrees of non-replicability in multiple scientific fields (15–19). In addition, reported results are frequently incorrect or misstated (20). Some commonly reported reasons include a lack of interoperability between software tools and data sets and the limited record-keeping maintained for complicated data sets, which impact coordinated analyses, reporting, and archiving. For example, crowdsourced analysis projects where multiple expert teams analyze the same data corpus reveal an enormous amount of analytical flexibility present in complex analyses of identical questions (e.g., 29 teams analyzed an identical dataset, with odds ratios for effects ranging from $0.89$ to $2.93$, $M=1.31$ (21)). Reproducibility of analyses require identical statistical analyses.

The reproducibility crisis is pronounced in the context of ANS data collection and analysis when using consumer sensors in real-world settings. The field lacks clear and standardized guidelines for analyzing autonomic data, leading to many disparate methods and a bottleneck in translating promising proofs-of-principle to widespread use. Researchers repeatedly build new algorithms and methods from scratch on new datasets without any meaningful comparisons with existing approaches or different datasets. This lack of benchmarking leads to a cycle of “reinventing the wheel” and “demonstrating feasibility.”

2 Envisioning an open-source, community-driven peripheral psychophysiological data processing framework

We envision an open-source framework that enables community users to create, execute, and share computational models and data analysis pipelines that address standardization, interpretation, and reproducibility challenges often encountered when analyzing ANS data. While several free and open-source software platforms are available for peripheral physiological data analysis (22–36), including some well-known, feature-rich software packages (e.g., WFDB (33) and openANSLAB (37)), we identified 10 common problem areas that impede broader acceptability and usability: (i) focus on only one or a few biosignals, each requiring its own analysis pipeline and signal-specific expertise; (ii) often download-on-request, or “freemium” (i.e., requiring payment for some analysis and input/output functions); (iii) designed for smaller laboratory datasets, not ambulatory datasets, which are typically much larger and require a significant investment of personnel time in error detection and correction; (iv) no integration across biosignals; (v) no explicit support for open scientific principles or platforms; (vi) unsupported, providing little or no accompanying documentation; (vii) inability to analyze the context in which data were collected; (viii) static, and not designed to incorporate code from other contributors; (ix) no ability to archive analysis pathways; and (x) command-line based, thus challenging for non-programmers. Proprietary software packages accompanying proprietary hardware and more general biosensor synchronization software have similar problems, as well as higher costs and closed code.

We developed a Survey of User Needs (SUN) to assess whether researchers and engineers from various scientific fields and disciplines who regularly process and analyze peripheral physiological and contextual data also experience the aforementioned analytical barriers. We circulated the survey to a large sample of researchers and engineers ($n=421$; $31\mathrm{\%}$ Researchers, $69\mathrm{\%}$ Engineers) from the Society of Psychophysiological Research, the IEEE International Machine Learning for Signal Processing Workshop, and snowball sampling using personal contacts and social media. Consistent with our review of existing software, over $70\mathrm{\%}$ of the researchers we surveyed confirmed facing difficulties syncing data from different sources, identifying errors in data, and combining data from different types of devices. They also reported that the various software tools were hard to learn and lacked clear instructions.

Our review of existing software and our survey results demonstrate that researchers and engineers across several disciplines working with multiple peripheral physiological signals would benefit from a multimodal data fusion platform with an open-source codebase. Indeed, SUN respondents were enthusiastic about a community-driven open-source framework that enables more transparency and reproducibility in the field. Based on this feedback, we outline the various core components we envision are needed.

2.1 Community driven

We envision that the sustainability of the framework can be achieved by inviting scientists and researchers to contribute state-of-the-art methodologies and algorithms as plugins. Researchers can contribute individual tasks like artifact removal or complete end-to-end pipelines and machine-learning models for a particular outcome. Such a collaborative approach will be crucial in addressing reproducibility challenges by (a) allowing engineers and computer scientists to validate and refine their approaches and algorithms and (b) allowing behavioral scientists and clinicians access to cutting-edge tools and methods for their work. By integrating these community-contributed plugins, we envision a dynamic and ever-evolving platform, always up to date with the latest advancements in the field.

To further ensure reproducibility, we anticipate researchers who design new methods and models to contribute their approaches to DBDP-Autonomic. The framework will include a foundational set of validated tools developed by our team, establishing a baseline for functionality. However, the goal is to build an iterative, community-driven ecosystem where contributions from other researchers play a central role. This approach benefits engineering and computer science researchers by increasing the adoption and visibility of their tools, while psychology and behavioral science researchers gain access to state-of-the-art techniques to analyze their data.

To maintain the reliability of the framework, contributed methods will undergo validation through benchmarking datasets and automated testing pipelines, ensuring compliance with established standards. Each plugin will include metadata specifying its purpose and validation outcomes, enabling transparency and usability. Over time, feedback from the community will help refine these tools, creating a continuous improvement cycle. This collaborative structure ensures that the framework remains robust and reproducible, advancing the field of psychophysiological research.

To facilitate these contributions, DBDP-Autonomic will provide standardized templates, clear documentation, and consistent input/output formats for all plugins. These templates will define common data structures and processing workflows, ensuring compatibility and ease of integration. This will enable researchers to contribute methods ranging from individual tasks, such as artifact removal, to comprehensive pipelines, while allowing users to seamlessly incorporate these plugins into their workflows. Furthermore, all generated syntax, results, visualizations, and meta-data should be documented. These could be stored locally and on a networked storage system available via a public interface to promote transparency and reproducibility. We envision that every step (with version control) associated with data processing and analysis, which we call the data supply chain (38), would be automatically saved as meta-data associated with a given dataset so other researchers can understand exactly how the data were collected, cleaned, and analyzed.

2.2 Data quality auditing and preprocessing

The framework should audit and assess the quality of peripheral physiological data collected in various settings by identifying and helping researchers address challenges like motion artifacts, environmental factors, and hardware limitations. To this end, we propose that the framework implement multiple semi-automated modules for data cleaning, preprocessing, and artifact removal, employing statistical and state-of-the-art machine-learning techniques as plugins (39–41). These software elements should cater to different data types and help researchers efficiently prepare and process their data for subsequent analysis.

2.3 Signal segmentation and alignment

A core aspect of ANS signal analysis is effective segmentation. Accordingly, the proposed framework should include a module that can determine appropriate time windows for signal segmentation based on the type of biosignal and research question. This module would help researchers and users select the optimal window length for their specific research needs, like short windows for time-domain features or longer ones for frequency-domain features. Aligning multimodal signals can be a challenge, particularly for uneven sampling rates, and tools can be included to support improved signal alignment (42).

2.4 Contextual information integration

Appropriately interpreting physiological data requires understanding the recording context, which includes characteristics of the environment outside the person (social, geolocation, ambient temperature, etc.) and those of the person (e.g., affect, physical activity, posture) that impact it. For instance, prior work demonstrates improved stress detection capabilities when physiological signals include contextual features (12). While several libraries allow researchers to do contextual processing (e.g., BeWie, RAPIDS), none integrate contextual information to drive physiological data processing, i.e., provide users with plots of physiological data with visual overlays that describe recording context (location change, physical activity, etc.) to help researchers determine whether contextual feature variables should be controlled for in subsequent analyses.

2.5 Data fusion and signal alignment

Researchers often use multiple peripheral physiological sensors and signals to study a particular outcome. Thus, data fusion and signal alignment are crucial. This step involves aligning data from different sensors or modalities, which might vary in sampling frequencies and timestamps. We envision community-contributed plugins capable of handling the intricacies of multimodal physiological data by harmonizing signals across different timescales and accommodating both short-term events and long-term trends.

2.6 Programming language and GUI

In the SUN survey, almost all behavioral scientists noted the need for a graphical user interface (GUI) to interact with open-source physiological processing tools. Thus, to cater to diverse user backgrounds and varying programming skills, the framework should be accessible both through a user-friendly GUI and a command-line interface (CLI). If common open programming languages (Python, R, Bash, etc.) are used, it would be convenient for researchers also to contribute their plugins and machine-learning models and use the framework to visualize their data and the outcomes of the various modules and plugins.

2.7 Science gateways and open science integration

The envisioned framework should align with Open Science Framework standards, ensuring compatibility with open science practices. The integration could include leveraging the Digital Health Data Repository, where researchers can easily share open-sourced, de-identified datasets while following relevant data management and reporting standards and considering privacy and ethical constraints.

3 DBDP autonomic

To effectuate our vision, we suggest expanding the Digital Biomarker Discovery Project¹ (DBDP (43)) to include dedicated processing of ANS signals – which we call DBDP Autonomic (Figure 2). DBDP is designed to serve as a hub for collaborative and open research in the field of digital health. Its current code repository includes computational building blocks for the most common measures of ambulatory physiological data collected through wearable devices, including photoplethysmography (PPG), electrocardiography (ECG), and electrodermal activity (EDA). The repository comprises four modules: (1) exploratory data analysis, (2) data preprocessing, (3) feature engineering, and (4) machine learning model development. Together, they provide users with methods needed to complete each component of an end-to-end data processing pipeline, including data cleaning and preprocessing tasks, analysis, and predictive model development.

Figure 2

Figure 2. DBDP Autonomic extends the functionalities of DBDP for biobehavioral research. (A) As signals from various modalities enter the analysis pipeline, DBDP Autonomic provides additional features on top of the existing modules in DBDP. These features extract and add contextual information, provide domain knowledge (for parameter tuning), and support multimodal data fusion. (B) Researchers can then integrate the processed features such as HR, RR, and BP to understand the autonomic constructs in the context of major domains of basic human neurobehavioral functioning. ECG, electrocardiogram; PPG, photoplethysmography; RIP, respiratory inductance plethysmography; BP, blood pressure; EDA, exploratory data analysis; HR, heart rate; HRV, heart rate variability; SBP, systolic BP; DBP, diastolic BP; SCR, skin conductance response; SCL, skin conductance level.

In addition to these general method modules, DBDP hosts an archive of code repositories and a list of open-source digital health data from internal and external collaborators. Contributors can upload their code to the DBDP archive or actively collaborate to update the methods repositories. DBDP is also developing a code-free GUI-based platform (DBDP Discovery) that enables users with little or no coding expertise to interact with the functionalities of DBDP modules either with datasets from the DHDR or their own data appropriately formatted (CSV, excel, etc.)

The uploaded code may remain standalone or be integrated into the main DBDP repository. Regardless of its location, it will adhere to the framework’s plugin guidelines, allowing it to be cloned as a submodule and used seamlessly alongside other DBDP components. This approach enables contributors to provide code for specific tasks (e.g., artifact removal) while allowing other users to incorporate these modules into their workflows without additional modifications. It functions similarly to how smaller modular ecosystems like React or npm packages operate, facilitating flexibility and ease of integration.

While general principles such as data fusion and signal alignment are broadly applicable to all physiological signals, DBDP-Autonomic specifically targets signals from the autonomic nervous system (ANS) to derive insights into psychological states. The ANS plays a critical role in processes like stress, arousal, and emotional regulation, making it uniquely positioned to bridge the physiological and psychological domains.

A key challenge in analyzing ANS data is the role of context. For instance, elevated heart rate and skin conductance might occur during both a stressful exam and a workout. While the physiological patterns may look similar, their psychological interpretations differ significantly. DBDP-Autonomic addresses this by integrating contextual data (e.g., activity type, location, and environmental factors) to disambiguate such scenarios and provide more accurate inferences about psychological states.

In addition, DBDP-Autonomic emphasizes the combination of multi-dimensional signals (e.g., heart rate variability, electrodermal activity, respiration) to create unified constructs that reflect psychological states more comprehensively. This approach contrasts with single-signal analysis, which often fails to capture the complexity of phenomena like stress or emotional regulation. Through contextual integration and multi-dimensional signal fusion, DBDP-Autonomic seeks to bridge critical gaps in understanding the interplay between physiology and psychology.

4 Discussion and call to community action

DBDP has established itself as an open-source hub for digital health, offering educational resources and computational tools for developing foundational features that collectively contribute to modeling complex biobehavioral outcomes. DBDP Autonomic, as an extension of this groundwork, could address the challenges associated with ANS signal analysis and enhance the standardization, interpretation, and reproducibility of existing and future research.

To advance biobehavioral research through DBDP Autonomic, we call upon the collective expertise of the digital health, behavioral, and psychophysiological research communities. Engagement and active participation from the community will be vital to ensuring the long-term viability and success of DBDP Autonomic. Our envisioned member engagement within DBDP Autonomic could follow the Center for Scientific Collaboration and Community Engagement (CSCCE) Community Participation Model (44), wherein multiple modes of interaction can coexist, with some members navigating through several nodes. Within this model, members typically initiate from the CONVEY/CONSUME mode, engaging with educational resources (e.g., tutorials and blog posts) to acquire biobehavioral and digital health knowledge. They may also access curated datasets and algorithms for their research. Transitioning to the CONTRIBUTE mode, research groups can add their go-to data cleaning algorithms, feature selection methods, and machine learning models as modules for other community members to use, benchmark, receive feedback, and cross-validate with other community-supplied methods. In the COLLABORATE mode, members of DBDP Autonomic could synergize and undertake joint research initiatives among the diverse community members who can apply, adapt, evaluate, and extend currently existing methods. Members could also co-author white papers and peer-reviewed publications to establish and enact standards in biobehavioral research. Lastly, members in the CO-CREATE mode could organize and lead workshops and working groups, driving the collective mission of DBDP Autonomic forward.

5 Conclusion

A collaborative effort of the DBDP Autonomic community could enable more robust, transparent, and reproducible research in biobehavioral health that involves ANS data. By emphasizing collaboration, transparency, and rigor, this resource could improve our understanding of complex biobehavioral health issues, provide personalized health insights, and accelerate the development of innovative interventions. As we move into the age of digital health, such a framework becomes essential for unlocking the full potential of mobile devices to benefit individual and community health.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

JD: Conceptualization, Writing – original draft, Writing – review & editing. VM: Conceptualization, Writing – original draft, Writing – review & editing. MMHS: Writing – original draft, Writing – review & editing. HJ: Writing – original draft, Writing – review & editing. NY: Writing – review & editing. YW: Writing – review & editing. BC: Writing – review & editing. MG: Conceptualization, Writing – original draft, Writing – review & editing.

Funding

The authors declare financial support was received for the research, authorship, and/or publication of this article. This research is partially supported by the National Institutes of Health, under awards P30DA029926, 5RO1LM014191, R01DK133531; the National Science Foundation, under award CAREER 2339669; and the American Heart Association, under award 23POST1025599. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the sponsors.

Acknowledgments

The authors would also like to thank the following colleagues for their insightful thoughts and discussions around this paper: Murat Akcakaya, Judith Andersen, Dana Brooks, Deniz Erdogmus, James Heathers, David Kaeli, Ian Kleckner, Sarah Ostadabbas, Karen Quigley, and Jolie Wormwood.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnote

1. ^https://www.dbdp.org/code-repositories

References

1. World Health Organization. Data from: World health statistics 2023: monitoring health for the SDGs, sustainable development goals (2023).

Google Scholar

2. Hovsepian K, Al’Absi M, Ertin E, Kamarck T, Nakajima M, Kumar S. cStress: towards a gold standard for continuous stress assessment in the mobile environment. In: Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp). ACM (2015). p. 493–504. doi: 10.1145/2750858.2807526

Crossref Full Text | Google Scholar

3. Mishra V, Pope G, Lord S, Lewia S, Lowens B, Caine K, et al. Continuous detection of physiological stress with commodity hardware. ACM Trans Comput Healthc. (2020) 1:1–30. doi: 10.1145/3361562

PubMed Abstract | Crossref Full Text | Google Scholar

4. Chikersal P, Doryab A, Tumminia M, Villalba DK, Dutcher JM, Liu X, et al. Detecting depression and predicting its onset using longitudinal symptoms captured by passive sensing. ACM Trans Comput Hum Interact. (2021) 28:1–41. doi: 10.1145/3422821

Crossref Full Text | Google Scholar

5. Wang R, Wang W, DaSilva A, Huckins JF, Kelley WM, Heatherton TF, et al. Tracking depression dynamics in college students using mobile phone and wearable sensing. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2018) 2:1–26. doi: 10.1145/3191775

PubMed Abstract | Crossref Full Text | Google Scholar

6. De Zambotti M, Cellini N, Goldstone A, Colrain IM, Baker FC. Wearable sleep technology in clinical and research settings. Med Sci Sports Exerc. (2019) 51:1538. doi: 10.1249/MSS.0000000000001947

PubMed Abstract | Crossref Full Text | Google Scholar

7. Vanoli E, Adamson PB, Lazzara R, Orr WC. Heart rate variability during specific sleep stages. Circulation. (1995) 91:1918–22. doi: 10.1161/01.CIR.91.7.1918

PubMed Abstract | Crossref Full Text | Google Scholar

8. Dart AM, Du X-J, Kingwell BA. Gender, sex hormones and autonomic nervous control of the cardiovascular system. Cardiovasc Res. (2002) 53:678–87. doi: 10.1016/S0008-6363(01)00508-9

PubMed Abstract | Crossref Full Text | Google Scholar

9. Moodithaya S, Avadhany ST. Gender differences in age-related changes in cardiac autonomic nervous function. J Aging Res. (2011) 2012:e679345. doi: 10.1155/2012/679345

PubMed Abstract | Crossref Full Text | Google Scholar

10. Siegel EH, Sands MK, Van den Noortgate W, Condon P, Chang Y, Dy J, et al. Emotion fingerprints or emotion populations? A meta-analytic investigation of autonomic features of emotion categories. Psychol Bull. (2018) 144:343–93. doi: 10.1037/bul0000128

PubMed Abstract | Crossref Full Text | Google Scholar

11. Wright RA, Kirby LD. Cardiovascular correlates of challenge and threat appraisals: a critical examination of the biopsychosocial analysis. Pers Soc Psychol Rev. (2003) 7:216–33. doi: 10.1207/S15327957PSPR0703_02

PubMed Abstract | Crossref Full Text | Google Scholar

12. Mishra V, Hao T, Sun S, Walter KN, Ball MJ, Chen C-H, et al. Investigating the role of context in perceived stress detection in the wild. In: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers - UbiComp ’18. ACM Press (2018). p. 1708–16. doi: 10.1145/3267305.3267537

Crossref Full Text | Google Scholar

13. Goodman SN, Fanelli D, Ioannidis JPA. What does research reproducibility mean?. Sci Transl Med. (2016) 8:341ps12. doi: 10.1126/scitranslmed.aaf5027

PubMed Abstract | Crossref Full Text | Google Scholar

14. Mishra V, Sen S, Chen G, Hao T, Rogers J, Chen C-H, et al. Evaluating the reproducibility of physiological stress detection models. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2020) 4(4):140. doi: 10.1145/3432220

Crossref Full Text | Google Scholar

15. Camerer CF, Dreber A, Forsell E, Ho T-H, Huber J, Johannesson M, et al. Evaluating replicability of laboratory experiments in economics. Science. (2016) 351:1433–6. doi: 10.1126/science.aaf0918

PubMed Abstract | Crossref Full Text | Google Scholar

16. Freedman LP, Gibson MC. The impact of preclinical irreproducibility on drug development. Clin Pharmacol Ther. (2015) 97:16–8. doi: 10.1002/cpt.9

PubMed Abstract | Crossref Full Text | Google Scholar

17. Hutson M. Artificial intelligence faces reproducibility crisis. Science. (2018) 359:725–6. doi: 10.1126/science.359.6377.725

PubMed Abstract | Crossref Full Text | Google Scholar

18. Open Science Collaboration. Estimating the reproducibility of psychological science. Science. (2015) 349:aac4716. doi: 10.1126/science.aac4716

PubMed Abstract | Crossref Full Text | Google Scholar

19. Sorkin BC, Kuszak AJ, Williamson JS, Hopp DC, Betz JM. The challenge of reproducibility and accuracy in nutrition research: resources and pitfalls. Adv Nutr. (2016) 7:383–9. doi: 10.3945/an.115.010595

PubMed Abstract | Crossref Full Text | Google Scholar

20. Bakker M, Wicherts JM. The (mis)reporting of statistical results in psychology journals. Behav Res Methods. (2011) 43:666–78. doi: 10.3758/s13428-011-0089-5

PubMed Abstract | Crossref Full Text | Google Scholar

21. Silberzahn R, Uhlmann EL, Martin DP, Anselmi P, Aust F, Awtrey E, et al. Many analysts, one data set: making transparent how variations in analytic choices affect results. Adv Methods Pract Psychol Sci. (2018) 1:337–56. doi: 10.1177/2515245917747646

Crossref Full Text | Google Scholar

22. Bach DR, Friston KJ. Model-based analysis of skin conductance responses: towards causal models in psychophysiology. Psychophysiology. (2013) 50:15–22. doi: 10.1111/j.1469-8986.2012.01483.x

PubMed Abstract | Crossref Full Text | Google Scholar

23. Bartels R, Neumamm L, Peçanha T, Carvalho ARS. SinusCor: an advanced tool for heart rate variability analysis. Biomed Eng Online. (2017) 16:110. doi: 10.1186/s12938-017-0401-4

PubMed Abstract | Crossref Full Text | Google Scholar

24. Benedek M, Kaernbach C. Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology. (2010) 47:647–58. doi: 10.1111/j.1469-8986.2009.00972.x

PubMed Abstract | Crossref Full Text | Google Scholar

25. Föll S, Maritsch M, Spinola F, Mishra V, Barata F, Kowatsch T, et al. FLIRT: a feature generation toolkit for wearable data. Comput Methods Programs Biomed. (2021) 212:106461. doi: 10.1016/j.cmpb.2021.106461

PubMed Abstract | Crossref Full Text | Google Scholar

26. Kaufmann T, Sütterlin S, Schulz SM, Vögele C. ARTiiFACT: a tool for heart rate artifact processing and heart rate variability analysis. Behav Res Methods. (2011) 43:1161–70. doi: 10.3758/s13428-011-0107-7

PubMed Abstract | Crossref Full Text | Google Scholar

27. Kleckner IR, Jones RM, Wilder-Smith O, Wormwood JB, Akcakaya M, Quigley KS, et al. Simple, transparent, and flexible automated quality assessment procedures for ambulatory electrodermal activity data. IEEE Trans Biomed Eng. (2018) 65:1460–7. doi: 10.1109/TBME.2017.2758643

PubMed Abstract | Crossref Full Text | Google Scholar

28. Mali B, Zulj S, Magjarevic R, Miklavcic D, Jarm T. Matlab-based tool for ECG and HRV analysis. Biomed Signal Process Control. (2014) 10:108–16. doi: 10.1016/j.bspc.2014.01.011

Crossref Full Text | Google Scholar

29. Nabian M, Nouhi A, Yin Y, Ostadabbas S. A biosignal-specific processing tool for machine learning and pattern recognition. In: 2017 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT). (2017). p. 76–80. doi: 10.1109/HIC.2017.8227588

Crossref Full Text | Google Scholar

30. Perakakis P, Joffily M, Taylor M, Guerra P, Vila J. KARDIA: a Matlab software for the analysis of cardiac interbeat intervals. Comput Methods Programs Biomed. (2010) 98:83–9. doi: 10.1016/j.cmpb.2009.10.002

PubMed Abstract | Crossref Full Text | Google Scholar

31. Pichot V, Roche F, Celle S, Barthélémy J-C, Chouchou F. HRVanalysis: a free software for analyzing cardiac autonomic activity. Front Physiol. (2016) 7:557. doi: 10.3389/fphys.2016.00557

PubMed Abstract | Crossref Full Text | Google Scholar

32. Ramshur J. Design, evaluation, and application of heart rate variability analysis software (HRVAS) (electronic theses and dissertations). (2010).

Google Scholar

33. Silva I, Moody GB. An open-source toolbox for analysing and processing physionet databases in MATLAB and octave. J Open Res Softw. (2014) 2:e27. doi: 10.5334/jors.bi

PubMed Abstract | Crossref Full Text | Google Scholar

34. Tarvainen MP, Niskanen J-P, Lipponen JA, Ranta-aho PO, Karjalainen PA. Kubios HRV – heart rate variability analysis software. Comput Methods Programs Biomed. (2014) 113:210–20. doi: 10.1016/j.cmpb.2013.07.024

PubMed Abstract | Crossref Full Text | Google Scholar

35. Vollmer M. A robust, simple and reliable measure of heart rate variability using relative RR intervals. In: 2015 Computing in Cardiology Conference (CinC). (2015). p. 609–12. doi: 10.1109/CIC.2015.7410984

Crossref Full Text | Google Scholar

36. Wagner J, Kim J, Andre E. From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: 2005 IEEE International Conference on Multimedia and Expo. (2005). p. 940–3. doi: 10.1109/ICME.2005.1521579

Crossref Full Text | Google Scholar

37. Blechert J, Peyk P, Liedlgruber M, Wilhelm FH. ANSLAB: integrated multichannel peripheral biosignal processing in psychophysiological science. Behav Res Methods. (2016) 48:1528–45. doi: 10.3758/s13428-015-0665-1

PubMed Abstract | Crossref Full Text | Google Scholar

38. Goldsack JC, Coravos A, Bakker JP, Bent B, Dowling AV, Fitzer-Attas C, et al. Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for biometric monitoring technologies (BioMeTs). npj Digit Med. (2020) 3:1–15. doi: 10.1038/s41746-020-0260-4

PubMed Abstract | Crossref Full Text | Google Scholar

39. Gashi S, Di Lascio E, Stancu B, Swain VD, Mishra V, Gjoreski M, et al. Detection of artifacts in ambulatory electrodermal activity data. Proc ACM Interact Mob Wearable Ubiquitous Technol. (2020) 4:44:1–44:31. doi: 10.1145/3397316

Crossref Full Text | Google Scholar

40. Kelsey M, Palumbo RV, Urbaneja A, Akcakaya M, Huang J, Kleckner IR, et al. Artifact detection in electrodermal activity using sparse recovery. Proc SPIE. (2017) 10211:102110D-1. doi: 10.1117/12.2264027

Crossref Full Text | Google Scholar

41. Taylor S, Jaques N, Weixuan Chen F, Picard R. Automatic identification of artifacts in electrodermal activity data. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). NIH Public Access (2015). Vol. 2015. p. 1934–7. doi: 10.1109/EMBC.2015.7318762

Crossref Full Text | Google Scholar

42. Jiang Y, Qi Y, Wang WK, Bent B, Avram R, Olgin J, et al. EventDTW: an improved dynamic time warping algorithm for aligning biomedical signals of nonuniform sampling frequencies. Sensors. (2020) 20:2700. doi: 10.3390/s20092700

PubMed Abstract | Crossref Full Text | Google Scholar

43. Bent B, Wang K, Grzesiak E, Jiang C, Qi Y, Jiang Y, et al. The digital biomarker discovery pipeline: an open-source software platform for the development of digital biomarkers using mhealth and wearables data. J Clin Transl Sci. (2020) 5:e19. doi: 10.1017/cts.2020.511

PubMed Abstract | Crossref Full Text | Google Scholar

44. Woodley L, Pratt K. The CSCCE community participation model—a framework to describe member engagement and information flow in STEM communities. (2020). doi: 10.5281/zenodo.3997802

Crossref Full Text | Google Scholar