Open Access Multimodal fNIRS Resting State Dataset With and Without Synthetic Hemodynamic Responses

Introduction

Functional Near-Infrared Spectroscopy (fNIRS) is an optical neuroimaging technology that has rapidly gained momentum within the last decades (Boas et al., 2014; Scholkmann et al., 2014; Yücel et al., 2017). It is a non-hazardous and non-invasive optical brain imaging technique that uses near-infrared light to measure local cortical concentration changes of oxygenated and deoxygenated hemoglobin (HbO₂/HbR), which are associated with brain metabolism (Villringer and Chance, 1997; Ferrari and Quaresima, 2012). fNIRS has been considered a cost-effective and mobile alternative for functional Magnetic Resonance Imaging in conventional neuroscientific research. It is very suitable—and thus increasingly being used—for single trial analysis and Brain Computer Interface (BCI) applications (Matthews et al., 2008; Hong et al., 2018) as a single modality or along with Electroencephalography (EEG). While EEG and fNIRS signal processing is essential to increase the contrast to noise ratio (CNR) of measured brain responses, the nature of the signals and processing methods differ greatly. Hemodynamic brain responses in fNIRS are usually masked by local and systemic physiological confounding signals, for instance from superficial (scalp) blood flow, low frequency oscillations (Mayer waves), motion and breathing (Elwell et al., 1999; Yücel et al., 2016; von Lühmann et al., 2019). New and increasingly complex and powerful statistical methods are being developed that aim to remove the confounding factors in the signal, improve CNR and increase the detection/classification accuracy of hemodynamic responses. An objective way of validating the power of these novel methods and comparing them with the existing ones is to use an fNIRS dataset which has all the confounding signals but also a known hemodynamic brain response. One solution for this problem is to generate realistic fNIRS ground truth data by modeling a hemodynamic response function (HRF) on top of real resting state data (Gagnon et al., 2012; von Lühmann et al., 2019, 2020a,b). This approach can be used as a good approximation for realistic fNIRS signals with evoked responses, for which the ground truth is available. Generating such data is comparatively straight forward but requires prior knowledge in fNIRS signal characteristics as well as experience in fNIRS signal processing. Moreover, the use of short-separation fNIRS measurements and additional physiological signals, such as accelerometer or photoplethysmography (PPG), has been shown to enable methods that yield improved CNR (Yücel et al., 2015; von Lühmann et al., 2020a), but there are only few openly available multimodal fNIRS datasets (Shin et al., 2017, 2018) and even fewer multimodal datasets that include sufficient resting state periods to enable the approach described above. Thus, as a remedy, here we provide such a multimodal dataset with (and without) added synthetic HRF ground truth, short-separation fNIRS measurements, accelerometer, and other physiological measurement, for the data science community in order to facilitate the validation of novel methods. We also provide a simple code example to enable customization and modification of the HRF ground truth in the data.

Methods

The resting state data consists of two subsets: Dataset I, with 5 min resting state data from 14 participants and Dataset II, with 10 min resting state data from 14 participants. The data details follow and are summarized in Figure 1.

FIGURE 1

Figure 1. Summary of dataset metadata. Red dots are fNIRS emitters, blue dots are fNIRS detectors. LS, Long-Separation; SS, Short-Separation; PPG, Photoplethysmography; RESP, Respiration; BP, Blood Pressure; ACCEL, Accelerometer. HRF with 20, 50, and 100 % of +0.66 | −0.23 μM peak HbO₂/HbR amplitude. Sensitivity profile in log10 dB.

Participants and Demographics

Dataset I consists of recordings from 14 healthy participants (age: 21 ± 2 years; 11 male/3 female) and Dataset II from 14 healthy participants (age: 32 ± 19 years; 7 male/6 female/1 not reported) with no neurological or psychological disorders.

Experimental Paradigm

Participants were seated in a comfortable chair with room light on and were asked to look at a fixation cross on a black screen ~50 cm in front of them. A 5-min resting state data (Dataset I) or a 10-min resting state data (Dataset II) were recorded from each participant.

Data Acquisition

fNIRS data were acquired using a multichannel continuous wave fNIRS system (CW6, TechEn Inc. MA, USA) operating at 690 and 830 nm wavelengths. The system is an optical imager with 32 frequency encoded lasers (half at 690 and half at 830 nm) and 32 avalanche photo-diode detectors. The light is carried from the system to the head probe and back via optical fiber bundles. fNIRS data were acquired at a sample rate of 50 Hz.

DATASET I: Optode Array and Auxiliary Measurements

Optode array

Both head optode arrays were designed utilizing AtlasViewer software (Aasted et al., 2015) (Figure 1). The optode array for Dataset I consisted of an elastic cap (EasyCap, Herrsching, Germany) with 8 sources, 12 long-separation detectors (~3 cm apart from the source) and 2 short-separation detectors (~1 cm apart from the source) providing, in total, 26 long-separation and 2 short-separation channels covering the occipital lobe.

Auxiliary measurements

Systemic physiological changes and head motions of the participants were simultaneously recorded along with the fNIRS data using an MP160 data acquisition and analysis system (BIOPAC Systems Inc., Goleta, CA). The pulse waveform was recorded using a PPG100C amplifier and TSD200 PPG pulse transducer placed on the participant's right index finger (BIOPAC Systems Inc., Goleta, CA). Respiration data was collected via measuring the abdominal (or thoracic) expansion and contraction using a RSP100C amplifier and a TSD201 respiration transducer (respiration belt) (BIOPAC Systems Inc., Goleta, CA) around the participant's chest. The blood pressure waveform was recorded using a DA100C amplifier and a TSD110 pressure transducer (BIOPAC Systems Inc., Goleta, CA) placed on the participant's right thumb. Head motions in x, y, z directions were collected using an accelerometer (ADXL335, Analog Devices Inc., Norwood, MA) secured on the head with a headband.

DATASET II: Optode Array and Auxiliary Measurements

Optode array

The optode array for Dataset II consisted of an elastic cap (EasyCap, Herrsching, Germany) with 16 sources, 24 long-separation detectors (~3 cm apart from the source) and 8 short-separation detectors (~1 cm apart from the source) providing, in total, 48 long-separation and 8 short-separation channels covering the head from frontal to parietal regions bilaterally.

Auxiliary measurement

Head motions of the participants in x, y, z directions were simultaneously recorded along with the fNIRS data using a 3-axis accelerometer (ADXL335, Analog Devices Inc., Norwood, MA) secured on the head with a headband.

Adding Synthetic HRF to the fNIRS Data

In the documented data repository, we provide the acquired resting state data with and without synthetic HRF as well as the scripts used for the generation of the data to enable users to alter and re-generate ground truth HRF according to their needs. We generate synthetic HRFs with three different amplitudes using a gamma function with a time-to-peak of 6 s and a total duration of 16.5 s. The shape of this synthetic HRF is also depicted in Figure 2A. The three amplitudes are provided as percentages (100/50/20%) of a typical average amplitude of a task-evoked HRF (Huppert et al., 2006) and simulate varying degrees of CNR in the data: The (100%) amplitude is equal to +1% | −2% change from baseline intensity at 690 nm | 830 nm leading to an HRF peak amplitude of +0.66 | −0.23 μM for HbO₂ | HbR, respectively with a differential pathlength factor of 6 (Delpy et al., 1988; Boas et al., 2004) for a 30 mm source-detector separation. For each participant in the two datasets, all resting state data is divided into windows of 20 s length. The HRFs are added in the intensity domain at a random onset (0–3.5 s) within each 20 s window for a randomly selected half of all available long separation channels after pruning with a 5 dB SNR threshold. This results in an average of 15 | 38 trials per participant and HRF amplitude in each resting state Dataset I | II.

FIGURE 2

Figure 2. Overview of data quality and baseline analysis for Datasets I and II. (A) Across-trial recovered HRF estimated with tCCA GLM using Homer3 for all three amplitudes (100, 50, 20%) (participant 33 from Dataset I and participant 98 from Dataset II - the first participant in each data set). Red: HbO₂, Blue: HbR, Black dashed: Ground Truth. (B) Mean SNR in dB across channels for all participants. Whiskers indicate standard deviation. (C) Mean Motion Ratio across channels for all participants which indicates the ratio of motion-contaminated data to the whole data. (D) Example time course of multimodal data in Dataset I. LS, Long-separation channel; SS, Short-separation channel; AccelX|Y|Z, Accelerometer; PPG, Photoplethysmogram; BP, Blood Pressure; RESP, Respiration.

Data Structure and Format

Both datasets are presented in the Shared Near Infrared File Format V1.0 Specification (snirf), which is based on the HDF5 file format (https://github.com/fNIRS/snirf). SnirfClass function loads the snirf object into the MATLAB environment. Table 1 provides the list of variables in the current dataset snirf object. The main fields of interest are: the data field which has the fNIRS raw signal at each channel and relevant information, the probe field which has optode array information, the aux field which has all the auxiliary measurements and their details, and the stim field which has the experimental paradigm information. Please note that, while snirf.aux(2)/(3)/(4) have AccelX, AccelY, and AccelZ measurements for both datasets, Dataset I has PPG, blood pressure (BP), RESP at snirf.aux(5), (6), and (7), respectively in addition to these.

TABLE 1

Table 1. Snirf object fields.

Baseline Analysis and Data Quality Assessment

Baseline analysis and data quality assessment was performed using the openly available Homer3 toolbox (https://github.com/BUNPC/Homer3) (Huppert et al., 2009). HRFs were recovered from the augmented resting state data using the processing stream provided in the repository (tCCA_xmpl_procStream_Gauss_noHPF.cfg under “code” folder). This processing stream includes 0.5 Hz zero phase low pass filter with an effective order of 6, conversion to HbO₂ and HbR using the modified Beer-Lambert Law (Delpy et al., 1988; Boas et al., 2004), and subsequent HRF estimation with the temporally embedded General Linear Model (tCCA GLM) approach using short-distance channels and a polynomial drift term for nuisance regression and Gaussian basis functions for the HRF regressor (von Lühmann et al., 2020a). Figure 2A exemplifies the resulting HRF estimates in one augmented channel for all three amplitudes from participant 33 from Dataset I and participant 98 from Dataset II. Data quality is provided for each participant as across-channel average of the Signal to Noise Ratio (SNR) in Figure 2B and as the mean motion ratio across channels in Figure 2C. Channel SNR is calculated as 20 × log10 of the mean over std. of the raw intensity signal. The motion ratio is calculated as the ratio between the cumulative time of segments in the data that were considered to be confounded by motion artifacts, as identified by the Homer2 function hmrMotionArtifactByChannel (with tMotion = 0.5, tMask = 0.5, STDEVthresh = 20, AMPthresh = 5), to the total acquisition time. Figure 2D displays a typical segment of all available signals (z-scored) in the first participant in Dataset I. Long and short-separation fNIRS channels exhibit typical low frequency components and cardiac pulsation, which is also present in the PPG and BP measurement.

Summary

We reported a multimodal fNIRS resting state dataset from 28 participants, that we provide with and without added synthetic HRF ground truth at three different amplitudes. We include the script used for the generation of these data to enable users to adapt this approach to their own needs. The availability of multiple auxiliary biosignals, such as motion (accelerometer) and PPG in the data, can be used to explore and extend existing multimodal fNIRS-based signal processing approaches (von Lühmann et al., 2019, 2020a). Resting fNIRS data with added known HRF enables the validation of novel processing methods for single trial HRF detection and BCI as well as more general artifact rejection and preprocessing approaches and their comparison with existing methods. This can also be useful for methods that tackle challenges such as non-stationarities in the amplitude and time to peak of hemodynamic responses to a stimulus.

Data Availability Statement

Publicly available datasets were analyzed in this study. This data and the code for HRF ground-truth generation can be found here: https://www.nitrc.org/frs/?group_id=1071.

Ethics Statement

The studies involving human participants were reviewed and approved by Boston University IRB. The participants provided their written informed consent to participate in this study.

Author Contributions

XL and NG collected the first and second datasets, respectively. AL and MY analyzed the data, prepared the figures, and wrote the report. DB reviewed the report. All authors confirmed the final version of the report.

Funding

This work was funded by a research contract under Facebook's Sponsored Academic Research Agreement and in part by NIH R24NS104096.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The authors declare that this study received funding from Facebook. The funder was involved in the decision to submit the paper for publication.

References

Aasted, C. M., Yücel, M., A., Cooper, R. J., Dubb, J., Tsuzuki, D., Becerra, L., et al. (2015). Anatomical guidance for functional near-infrared spectroscopy: AtlasViewer tutorial. Neurophotonics 2:020801. doi: 10.1117/1.NPh.2.2.020801

PubMed Abstract | CrossRef Full Text | Google Scholar

Boas, D. A., Dale, A. M., and Franceschini, M. A. (2004). Diffuse optical imaging of brain activation: approaches to optimizing image sensitivity, resolution, and accuracy. Neuroimage 23, 275–288. doi: 10.1016/j.neuroimage.2004.07.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Boas, D. A., Elwell, C. E., Ferrari, M., and Taga, G. (2014). Twenty years of functional near-infrared spectroscopy: introduction for the special issue. Neuroimage 85, 1–5. doi: 10.1016/j.neuroimage.2013.11.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Delpy, D. T., Cope, M., van der Zee, P., Arridge, S., Wray, S., and Wyatt, J. (1988). Estimation of optical pathlength through tissue from direct time of flight measurement. Phys. Med. Biol. 33:1433. doi: 10.1088/0031-9155/33/12/008

PubMed Abstract | CrossRef Full Text | Google Scholar

Elwell, C. E., Springett, R., Hillman, E., and Delpy, D. T. (1999). Oscillations in cerebral haemodynamics. Implications for functional activation studies. Advances in. Experimental Exp. Medicine and. Biology Biol. vol. 471, 57–65. doi: 10.1007/978-1-4615-4717-4_8

PubMed Abstract | CrossRef Full Text | Google Scholar

Ferrari, M., and Quaresima, V. (2012). A brief review on the history of human functional near-infrared spectroscopy (fNIRS) development and fields of application. Neuroimage 63, 921–935. doi: 10.1016/j.neuroimage.2012.03.049

PubMed Abstract | CrossRef Full Text | Google Scholar

Gagnon, L., Cooper, R. J., Yücel, M. A., Perdue, K. L., Greve, D. N., and Boas, D. A. (2012). Short separation channel location impacts the performance of short channel regression in NIRS. Neuroimage 59, 2518–2528. doi: 10.1016/j.neuroimage.2011.08.095

PubMed Abstract | CrossRef Full Text | Google Scholar

Hong, K.-S., Khan, M. J., and Hong, M. J. (2018). Feature extraction and classification methods for hybrid fNIRS-EEG brain-computer interfaces. Front. Hum. Neurosci. 12:246. doi: 10.3389/fnhum.2018.00246

PubMed Abstract | CrossRef Full Text | Google Scholar

Huppert, T. J., Diamond, S. G., Franceschini, M. A., and Boas, D. A. (2009). HomER: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl. Opt. 48, 280–298. doi: 10.1364/AO.48.00D280

PubMed Abstract | CrossRef Full Text | Google Scholar

Huppert, T. J., Hoge, R. D., Diamond, S. G., Franceschini, M. A., and Boas, D. A. (2006). A temporal comparison of BOLD, ASL, and NIRS hemodynamic responses to motor stimuli in adult humans. Neuroimage 29, 368–382. doi: 10.1016/j.neuroimage.2005.08.065

PubMed Abstract | CrossRef Full Text | Google Scholar

Matthews, F., Pearlmutter, B. A., Ward, T. E., Soraghan, C., and Markham, C. (2008). Hemodynamics for brain-computer interfaces. Signal Process. Mag. IEEE 25, 87–94. doi: 10.1109/MSP.2008.4408445

CrossRef Full Text | Google Scholar

Scholkmann, F., Kleiser, S., Metz, A. J., Zimmermann, R., Mata Pavia, J., Wolf, U., et al. (2014). A review on continuous wave functional near-infrared spectroscopy and imaging instrumentation and methodology. Neuroimage 85, 6–27. doi: 10.1016/j.neuroimage.2013.05.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Shin, J., von Lühmann, A., Blankertz, B., Kim, D.-W. W., Jeong, J., Hwang, H.-J. J., et al. (2017). Open access dataset for EEG+ NIRS single-trial classification. IEEE Trans. Neural Syst. Rehabil. Eng. 25, 1735–1745. doi: 10.1109/TNSRE.2016.2628057

CrossRef Full Text | Google Scholar

Shin, J., Von Lühmann, A., Kim, D. W., Mehnert, J., Hwang, H. J., and Müller, K. R. (2018). Data descriptor: simultaneous acquisition of EEG and NIRS during cognitive tasks for an open access dataset. Sci. Data 5:180003. doi: 10.1038/sdata.2018.3

PubMed Abstract | CrossRef Full Text | Google Scholar

Villringer, A., and Chance, B. (1997). Non-invasive optical spectroscopy and imaging of human brain function. Trends Neurosci. 20, 435–442. doi: 10.1016/S0166-2236(97)01132-6

PubMed Abstract | CrossRef Full Text | Google Scholar

von Lühmann, A., Boukouvalas, Z., Müller, K. R., and Adal,i, T. (2019). A new blind source separation framework for signal analysis and artifact rejection in functional near-infrared spectroscopy. Neuroimage 200, 72–88. doi: 10.1016/j.neuroimage.2019.06.021

PubMed Abstract | CrossRef Full Text | Google Scholar

von Lühmann, A., Li, X., Müller, K. R., Boas, D. A., and Yücel, M. A. (2020a). Improved physiological noise regression in fNIRS: a multimodal extension of the General Linear Model using temporally embedded Canonical Correlation Analysis. Neuroimage 208:116472. doi: 10.1016/j.neuroimage.2019.116472

PubMed Abstract | CrossRef Full Text | Google Scholar

von Lühmann, A., Ortega-Martinez, A., Boas, D. A., and Yücel, M. A. (2020b). Using the general linear model to improve performance in fnirs single trial analysis and classification: a perspective. Front. Hum. Neurosci. 14, 1–1730. doi: 10.3389/fnhum.2020.00030

PubMed Abstract | CrossRef Full Text | Google Scholar

Yücel, M. A., Selb, J., Aasted, C. M., Lin, P.-Y., Borsook, D., Becerra, L., et al. (2016). Mayer waves reduce the accuracy of estimated hemodynamic response functions in functional near-infrared spectroscopy. Biomed. Opt. Express 7, 3078–3088. doi: 10.1364/BOE.7.003078

PubMed Abstract | CrossRef Full Text | Google Scholar

Yücel, M. A., Selb, J., Aasted, C. M., Petkov, M. P., Becerra, L., Borsook, D., et al. (2015). Short separation regression improves statistical significance and better localizes the hemodynamic response obtained by near-infrared spectroscopy for tasks with differing autonomic responses. Neurophotonics 2:035005. doi: 10.1117/1.NPh.2.3.035005

PubMed Abstract | CrossRef Full Text | Google Scholar

Yücel, M. A., Selb, J. J., Huppert, T. J., Franceschini, M. A., and Boas, D. A. (2017). Functional near infrared spectroscopy: enabling routine functional brain imaging. Curr. Opin. Biomed. Eng. 4, 78–86. doi: 10.1016/j.cobme.2017.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar