Toward Predicting Human Performance Outcomes From Wearable Technologies: A Computational Modeling Approach

Wearable technologies for measuring digital and chemical physiology are pervading the consumer market and hold potential to reliably classify states of relevance to human performance including stress, sleep deprivation, and physical exertion. The ability to efficiently and accurately classify physiological states based on wearable devices is improving. However, the inherent variability of human behavior within and across individuals makes it challenging to predict how identified states influence human performance outcomes of relevance to military operations and other high-stakes domains. We describe a computational modeling approach to address this challenge, seeking to translate user states obtained from a variety of sources including wearable devices into relevant and actionable insights across the cognitive and physical domains. Three status predictors were considered: stress level, sleep status, and extent of physical exertion; these independent variables were used to predict three human performance outcomes: reaction time, executive function, and perceptuo-motor control. The approach provides a complete, conditional probabilistic model of the performance variables given the status predictors. Construction of the model leverages diverse raw data sources to estimate marginal probability density functions for each of six independent and dependent variables of interest using parametric modeling and maximum likelihood estimation. The joint distributions among variables were optimized using an adaptive LASSO approach based on the strength and directionality of conditional relationships (effect sizes) derived from meta-analyses of extant research. The model optimization process converged on solutions that maintain the integrity of the original marginal distributions and the directionality and robustness of conditional relationships. The modeling framework described provides a flexible and extensible solution for human performance prediction, affording efficient expansion with additional independent and dependent variables of interest, ingestion of new raw data, and extension to two- and three-way interactions among independent variables. Continuing work includes model expansion to multiple independent and dependent variables, real-time model stimulation by wearable devices, individualized and small-group prediction, and laboratory and field validation.

Introduction

Advances in wearable sensing afford real-time non-invasive monitoring of digital and chemical physiology, behavior, and biomechanics in ambulatory individuals (Windmiller and Wang, 2013; Lou et al., 2018, 2020). Wearable consumer and medical devices can collect real-time data from diverse locations on the body (e.g., wrist, finger, chest, head, and oral cavity) and process and classify data to predict user status (Nag et al., 2017; Tseng et al., 2018; Terse-Thakoor et al., 2020). Classification can occur across one or more dimensions including stress, fatigue, sleep status, thermal load, hydration, blood glucose and oxygenation, and physical exertion (Yoo and Lee, 2010; Coppedè et al., 2014; Mokaya et al., 2016; Kutilek et al., 2017; Dunican et al., 2018; Ahn et al., 2019). In military or other high-stakes contexts, real-time readings of these outputs can provide trainers and leaders with information regarding the current physiological and behavioral status of individuals and teams (Kutilek et al., 2017; Friedl, 2018; Scheit, 2021).

Awareness of individual or group status in a given domain (e.g., heart rate) is not always sufficient for informing predictions of human performance outcomes, making it challenging to bridge the gap between state classification, what the status predicts for cognitive and physical performance, and what can be done to mitigate any predicted performance degradation (Danah and Crawford, 2012; Nafus, 2014; National Research Council, 2015; Howell et al., 2018; Parsons et al., 2019). For example, while actigraphy-based classification of sleep states might suggest low sleep quality or quantity, leaders may not understand how such states influence performance outcomes and therefore how to manage or mitigate the situation. Indeed, the relationship between sleep loss and performance outcomes is highly complex (Killgore, 2010). Similarly, saliva-based biomarkers of cortisol response via real-time tooth-borne sensing might suggest a high stress state (Singh et al., 2014; Tseng et al., 2018; Upasham et al., 2018), but the precise relationship between increased stress and performance outcomes remains elusive (Sandi, 2013).

The apparent disconnect between sensor outputs, status, and performance outcomes exists for at least three reasons. First, human cognitive and physical performance shows high variability within and across individuals, making performance challenging to reliably capture, model and predict, ultimately limiting the reliability of model outcomes (Smith et al., 2014). Second, computational modeling in this domain tends to be restricted to relatively few predictors and outcomes of interest and is computed using data from one or only a small handful of studies, ultimately limiting the applicability of model outcomes (Scheinost et al., 2019). Third, most research examining links between user states and cognitive and physical behavior is conducted in laboratory settings and may bear little resemblance to the tasks, environments, and behaviors characterizing relatively real-world settings, ultimately limiting the generalizability of model outcomes (Swets and Bjork, 1990; Terrin et al., 2003; Vergouwe et al., 2010).

To motivate our computational modeling approach to the challenge of human performance prediction, we first describe research on our three predictors of interest: stress, sleep, and physical exertion. Next, we describe research on our three outcomes of interest: reaction time, executive function, and perceptuo-motor control. We then describe our computational modeling approach, its outcomes, and its strengths and weaknesses. Finally, we discuss how our model framework can be extended to more complex prediction challenges, integrated with real-time state classification tools, and validated in laboratory and field research settings.

Predictors of Interest: Stress, Sleep, and Physical Exertion

We chose three predictors of interest based on emerging wearable technologies and their ability to classify stress states, sleep status, and extent of physical exertion. For example, tooth-borne biosensors can monitor saliva in the oral cavity to measure alpha amylase and estimate stress states (Robles et al., 2011; Tseng et al., 2018), wrist-worn accelerometers can monitor actigraphy and estimate sleep/wake cycles, including sleep loss and deprivation (Dunican et al., 2018), and arm- or leg-worn mechanomyography and electromyography sensors can be used to characterize the intensity and duration of physical exertion (Esposito et al., 1998; Woodward et al., 2019). Given the increasing availability of these sensors, these three specific predictors are likely to be used in real-world situations.

Stress is a normal response to physiologically and emotionally challenging experiences, triggered by situations that are inherently novel, uncontrollable, socially threatening, or uncertain (Gagnon and Wagner, 2016). The brain is a critical component of a stress response in two ways; first, because it is the source of the stress response itself (i.e., the organ responsible for determining whether an experience is threatening), and second, because it is also responsible for activating physiological and behavioral responses to stress (McEwen, 2007). At a basic physiological level, stress activates the autonomic nervous system and hypothalamic-pituitary-adrenal (HPA) axis, and carries diverse neurotransmitter, hormonal, genomic, and immune implications (Axelrod and Reisine, 1984; Padgett and Glaser, 2003; Charmandari et al., 2005; McEwen, 2007; Gagnon and Wagner, 2016; Buchheim et al., 2019; Angelova et al., 2021). When an individual is exposed to a stressor, two stress systems are activated: a rapid catecholamine response, and a slower HPA axis and genomic response. The rapid catecholamine response is associated with increased epinephrine and norepinephrine release (Schwabe et al., 2012), and the slower HPA response is associated with increased glucocorticoid release that induces both non-genomic and genomic effects on the central nervous system (Sapolsky et al., 2000). The effects of these stress responses are most pronounced on brain regions with high catecholamine and glucocorticoid receptor densities, including the amygdala, striatum, hippocampus, and prefrontal cortex (Arnsten, 1998, 2009; Roozendaal et al., 2009; Kim et al., 2015). Given the diverse perceptual, cognitive, and affective functions of these brain areas, stress correspondingly carries diverse implications for sustaining mental performance. For example, research shows that stress can positively influence performance on relatively simple or well-rehearsed tasks (Broadbent, 1971; Arnsten, 2009) that do not demand prefrontal cortical engagement, but can negatively influence performance on relatively difficult tasks involving high levels of executive function (Arnsten, 1998, 2009; Cerqueira et al., 2007), which can be especially pronounced, for example during unpredictable and ambiguous military operations (Gaillard et al., 2006).

Sleep plays a critical role in sustaining life and health, with acutely or chronically disrupted sleep associated with diverse effects on brain function and systemic physiology, including hormonal, cardiovascular, immune, and metabolic effects (Medic et al., 2017). Unfortunately, most American adults report trouble falling asleep, poor quality sleep, or difficulty staying asleep (Knutson et al., 2017). Military work schedules exacerbate this issue, with chronic demands for continuous operations minimizing the quantity and quality of sleep (Mysliwiec et al., 2013). These include 24-hour work cycles, time zone changes, inconsistent lighting exposure, and night shifts, which can cause recurrent partial sleep deprivation and/or acute total sleep deprivation (Spiegel et al., 2016). Due to these challenging demands, one study demonstrated that 72% of service members reported six or fewer hours of nightly sleep, 43% reported five or fewer hours, and 18% reported four or fewer hours (Luxton et al., 2011). These statistics can be compounded by clinical disorders such as insomnia, post-traumatic stress disorder, and mild traumatic brain injury (mTBI), and carry important implications for cognitive and physical performance (Good et al., 2020). Restricted sleep quantity and quality influence blood glucose and glucose tolerance, limit immune function, increase blood levels of catecholamines and cortisol, and attenuate brain functional connectivity in regions known to modulate attention (Dinges et al., 1995; Spiegel et al., 1999; Chee and Tan, 2010; Lim et al., 2010; Besedovsky et al., 2012). While some debate exists regarding the robustness and reliability of sleep restriction or deprivation effects on various aspects of cognitive performance, it is generally accepted that it negatively influences vigilance and sustained attention (i.e., the ability to attend and respond to stimuli; Lim and Dinges, 2008), and induces high performance variability (Doran et al., 2001).

Physical exertion describes sustained physical activity at a moderate to high intensity level. According to the American College of Sports Medicine (ACSM) and Centers for Disease Control and Prevention (CDC), physical activity can be generally classified by the rate of energy expenditure: low, moderate, and vigorous (Ainsworth et al., 1993, 2000, 2011). One method for measuring rate of energy expenditure over time is through metabolic equivalents (METs), which are calculated as working metabolic rate relative to resting metabolic rate (Byrne et al., 2005). As METs increase over extended durations, an increasingly diverse set of metabolic, hormonal, and neurotransmitter effects occurs, ultimately resulting in fatigue and exhaustion. The cognitive responses that occur during or following an acute bout of physical exertion have been attributed to a wide range of physiological responses including cardiac output, reticular activation, catecholamine and glucocorticoid levels, cerebral blood oxygenation, and brain-derived neurotrophic factor (Chang et al., 2012). While many studies have been reviewed and subjected to meta-analysis to describe cognitive effects after exercise (Etnier et al., 1997; Sibley and Etnier, 2003; Lambourne and Tomporowski, 2010), relatively few have considered these effects during exercise (Chang et al., 2012). During moderate or high levels of physical exertion, studies suggest that cognitive performance is weakly but negatively affected, particularly for perceptual and processing speed tasks (Lambourne and Tomporowski, 2010). A more recent and comprehensive review and meta-analysis suggested a much more complex relationship between exertion and cognitive performance, with certain tasks being negatively and others positively related to exertion, further influenced by exercise intensity (Chang et al., 2012).

Outcomes of Interest: Reaction Time, Executive Function, and Perceptuo-Motor Control

Cognitive processes are generally divided into two levels: low-level and higher-order (Miller and Cohen, 2001; Miller and Wallis, 2009). First, relatively low-level processes involve basic sensory, motor, and memory demands, and relatively routine habits and skills. Second, relatively higher-order processes involve more specialized executive functions that enable the control of attention, maintenance of task- and goal-related information, inhibition, and flexible and creative thought. We selected three outcomes of interest that represent both low-level and higher-order processes relevant to sustained performance on real-world tasks: reaction time, executive function, and perceptuo-motor control.

Reaction time is the latency from the presentation of a stimulus to one or more sensory modalities (e.g., visual, auditory), and a behavioral response (e.g., button press; Welford, 1980; Luce, 1986). Most perceptual and cognitive science experiments measuring reaction times use one of three tasks: simple reaction time, choice reaction time, and serial reaction time tasks (Kosinski, 2010). Simple reaction time tasks involve the presentation of a single stimulus with a single possible response; for example, repeatedly presenting a dot on a computer screen and asking participants to respond as quickly as possible to each dot presentation. Choice reaction time tasks involve the presentation of more than one stimulus, and response types are mapped to each stimulus; for example, presenting a dot or square on a screen and asking a participant to respond by pressing the number 1 or 3 key on a keyboard, respectively, in response to the presentation of each stimulus. Finally, serial reaction time tasks represent a modification of the choice reaction time task, including predictable sequences of each stimulus type. We chose to restrict our analysis to simple reaction time as an outcome of interest for three primary reasons. First, simple reaction time resides at a most basic level of cognitive processes, whereas choice and serial reaction time tasks can elicit higher-order processing (i.e., executive function) as evidenced by activation of inhibitory processes and anterior cingulate and prefrontal cortical circuits (Naito et al., 2000; Mulert et al., 2003; Burle et al., 2004). Second, maintaining vigilant attention and quickly reacting to emerging events is critical for sustained performance in a number of high-stakes tasks including those characteristic of military and first-responder operations (Truszczynski et al., 2014; Vrijkotte et al., 2016; Sheffield et al., 2017; Dominski et al., 2018; Brunyé et al., 2020). Finally, reaction time is well-characterized and widely available for efficient download and integration into our modeling approach, and has proven sensitive to variations in acute stress (Lieberman et al., 2002; Olver et al., 2015), sleep loss (Forest and Godbout, 2000; Bartel et al., 2004; Deslandes et al., 2006), and physical exertion (Chang et al., 2012).

Executive function (also referred to as cognitive control) involves developing, flexibly executing, and updating goals and strategies to coordinate performance on relatively complex tasks (Logan, 1985; Miyake et al., 2000). As a higher-order brain function, performing executive functions such as updating goals and inhibiting behavior tends to recruit a very diverse set of cortical and subcortical brain regions including the anterior cingulate, prefrontal cortex, and parietal lobes (Miller and Cohen, 2001; Curtis and D’Esposito, 2003; Seeley et al., 2007; Elton and Gao, 2014). Example executive function tasks typically used in laboratory settings include working memory tasks, the Stroop task, flanker tasks, go/no-go and stop-signal tasks, and problem solving tasks (Donders, 1969; Eriksen and Eriksen, 1974; Rabbitt, 2004; MacLeod, 2005; Lipszyc and Schachar, 2010). We chose to restrict our analysis to executive function tasks eliciting inhibitory control, for two primary reasons. First, inhibitory control is a crucial executive function that has been extensively linked to brain networks and highly relevant performance outcomes such as making shoot/don’t-shoot decisions in law enforcement and military contexts (Biggs et al., 2015; Scribner, 2016; Biggs, 2021). Second, inhibitory control processes are effectively isolated from simple reaction time both behaviorally and neuroanatomically, with largely independent functional brain networks responsible for their performance (Seeley et al., 2007; Elton and Gao, 2014). Finally, inhibitory control has proven sensitive to acute stress (Shields et al., 2016), sleep loss (Drummond et al., 2006; Killgore et al., 2009), and physical exertion (Dietrich, 2006; Dietrich and Audiffren, 2011; Eddy et al., 2015).

Perceptuo-motor control involves the dynamic interaction between sensory and perceptual systems with motor outputs and feedback and is a critical element of adaptive goal-oriented behavior at the level of single effectors (finger, tongue), limbs (arm, leg), and whole-body coordinated movement (Trommershäuser et al., 2003; Verhagen et al., 2008). Examples include dynamically adapting the aim of a weapon relative to a moving target, the swing of a tennis racquet relative to an oncoming ball, or strategically positioning of your feet on a balance beam. Many experiments examining perceptuo-motor control tend to use relatively simple trajectories of single limbs, such as reaching movements, and the control processes involved are thought to maximize goal completion while minimizing biomechanical costs (Sabes and Jordan, 1997; Trommershäuser et al., 2003). From a physiological perspective, controlling body movements involves a dynamic interaction among multiple subcorical and cortical brain regions involved in motor planning and execution (premotor and motor regions), visual perception (somatosensory and occipital regions), and error correction (inferior frontal and cerebellar regions; Gazzola and Keysers, 2009; Tanaka et al., 2009). We chose to include perceptuo-motor control as an outcome variable of interest for two primary reasons. First, perceptuo-motor control underlies myriad tasks in high-stakes contexts including weapons use, driving and aviation, surgical procedures, and firefighting (Pick and Palmer, 1986; Holden et al., 1999; Palmer et al., 2013; Louw et al., 2017). Second, the successful performance of complex perceptuo-motor tasks has proven sensitive to variations in acute stress (Arora et al., 2010b; Bajunaid et al., 2017), sleep loss (Jackson et al., 2013; Connaboy et al., 2020), and physical exertion (Tharion et al., 1997; Grebot et al., 2003).

Modeling Approach

We developed a model for the conditional probability density function (PDF) of each outcome, here denoted y_i, i ∈ {1, 2, 3}, given the three predictors, x_j for j ∈ {1, 2, 3};¹ which, by the definition of conditional probability, is computed as

$$\begin{array}{c}p\left(y_i|x_1,x_2,x_3\right)=\frac{p(y_i,x_1,x_2,x_3)}{p(x_1,x_2,x_3)}\hfill \\ =\frac{p(y_i,x_1,x_2,x_3)}{\int p(y_i,x_1,x_2,x_3)\text{d}{y}_i}\hfill \end{array}(1)$$

where p(y_i, x₁, x₂, x₃) is the joint distribution of the i-th outcome and the three predictors, p(x₁, x₂, x₃) is the joint distribution of the three predictors alone.

We note that the conditional distribution provides a complete probabilistic model for the outcome given the three predictors. From this model any statistical quantity of interest can be computed including but not limited to mean and standard deviation, mode (the most likely outcome to be sampled from a distribution), median, percentiles, as well as higher order moments such as skew and kurtosis. This characteristic of the model is helpful when users attempt to derive estimates of model confidence, for instance by relying on standard deviation and kurtosis measures.

Constructing the model requires that we determine the p(x_i), the marginal distributions for the predictors, as well as the joint distribution p(y_i, x₁, x₂, x₃). As detailed shortly, we also require p(y_i), the marginals for the outcomes for determining the joint distribution. We compute p(x_i) and p(y_i) using density estimation methods applied to data acquired from a variety of original research data sources. Using these estimated marginals as well as effect sizes gleaned from the literature, an optimization-based approach is developed which determines a joint distribution which “best” fits the marginals and the moment information implied by effect sizes. Both processes are detailed below.

Gathering Model Inputs: Marginal Distributions

For each of the identified predictors and outcomes we estimated PDFs to characterize the marginal distributions. These PDFs serve as inputs into our model. This process involved two primary steps, which we describe in turn.

First, we identified, downloaded, and organized open access data for each of the six variables of interest. To identify extant data, we leveraged longitudinal study databases, primary research reports that included supplementary data alongside the publication and contacted colleagues and authors to request archival data. Table 1 provides a full list of data sources used to define marginal distributions. Table 2 details the primary measures that were identified for each of the six variables of interest, including the number of data points identified. When identifying the primary measures for each variable, we chose to select gold-standard measures that could feasibly be measured and characterized in both laboratory (seated, still) and field (moving, sweating) conditions. For example, when selecting measures of stress, we chose to include measures that could be sensed with saliva sensing (i.e., cortisol, alpha amylase), physiological sensing (i.e., heart rate), and subjective stress ratings. In contrast, we chose to omit some specialized measures such as electrodermal activity (EDA), which may be valuable for detecting acute stress states in laboratory settings (Liu and Du, 2018) but relatively limited or challenging to interpret under conditions of thermoregulatory sweating and intense movement (Boucsein et al., 2012; Posada-Quintero et al., 2018). When similar measures (e.g., cortisol) used different measurement units, we converted to a common unit. To calculate error rates for executive function and perceptuo-motor control, we calculated a moving average for each of five successive trials in an original data set.

TABLE 1

Table 1. Original resources for deriving raw data to define marginal distributions (upper row) and effect size estimates to define xy relationships (lower row).

TABLE 2

Table 2. The six variables of interest, including three predictors and three outputs, their primary methods of measurement, and the number of data points used to estimate each variable’s PDF.

Second, we estimated PDFs from the data we collected. We used two different methods, depending on whether the data for a variable could be converted to a common unit or not.

All Units the Same – Non-parametric Density Estimation

For variables whose datapoints can all be converted to a common unit (sleep, physical exertion, reaction time, executive function, and perceptuo-motor control – see Table 1 for the primary measure for each variable), nonparametric density estimation methods (Izenman, 1991) are employed to construct the PDF. To be concrete, suppose we have N data sources for y₁. Then the PDF p(y₁) is computed as

$$p(y_1)=\sum _{k=1}^N{\mathrm{\pi }}_{k}p(y_1|F_k)$$

where F_k denotes the k-th data source, π_k is the fraction of points from the N sources coming from F_k, and p(y₁|F_k) is the conditional PDF of y₁ under the conditions associate with F_k. This function is computed and manually tuned from the raw data using the kde function from scikit-learn (Pedregosa et al., 2011).

Different Units – Mixture Modeling and Maximum Likelihood

For the one variable with multiple primary measures (stress, derived from measures of cortisol, alpha amylase, heart rate, and subjective responses), we needed to map disparate data into a common space. We will illustrate the process in this section. While stress is mentioned here, the process can be applied to any future variable with multiple primary measures.

Assume we have N sources of information for stress, F_i, i = 1, 2, …, N. Let d_i be a random variable (in the probabilistic sense) for the data from the i-th source. We assume that these data are related to the latent variable stress, s, via unknown functions, g_i(s) which for simplicity in this manuscript we take as affine. Thus, the j-th data point from the i-th source is

$$d_{i,j}=g_i\left(s_j\right)=a_i{s}_{i,j}+b_i$$

With θ_i = {a_i, b_i} the parameters associated with the model for the i-th data source, basic properties of derived random variables (Yates and Goodman, 2014) relates the PDF of the observed data given these parameters to the PDF of s as

$$p\left(d|{\mathrm{\theta }}_i\right)=\frac{1}{\left|a_i\right|}{p}_S\left(\frac{d-b_i}{a_i}\right)(2)$$

In (2) above, the notation $p_S\left(\frac{d-b_i}{a_i}\right)$ indicates the PDF of stress evaluated at the point (d − b_i)/a_i. For p_S(s), we use a mixture of Gaussians

$$p_S\left(s\right)=\sum _{c=1}^{N_c}{\mathrm{\alpha }}_{c}g(s;{\mathrm{\mu }}_c,{\mathrm{\sigma }}_c^2)$$

where g(x; a, b) denotes a Gaussian PDF in the variable x with mean a and variance b and for c = 1, 2, …, N_c we have 0 ≤ α_c ≤ 1, ∑_c α_c = 1, σ_c > 0, and μ_c is any real number. In the remainder of this manuscript, we used N_c = 6 with good results. We leave for future work the task of choosing this parameter in an adaptive manner.

Given the data from all N sources then, the problem we face is to find the maximum likelihood (ML) estimates of the following parameters:

$$\text{Θ}\equiv \{a_i,b_i,{\mathrm{\alpha }}_c,{\mathrm{\mu }}_c,{\mathrm{\sigma }}_c\}\text{for}i=1,2,\mathrm{\dots },N,c=1,2,\mathrm{\dots },N_c.$$

To find the ML estimate, we need the joint distribution of the data. Here we assume that all data points are independent draws from their respective conditional distributions, p(d_i|θ_i). We let N_i be the number of observed datapoints from source i and d_{i, j} the k-th observed datum from source i where j = 1, 2, …, N_i. With d the vector of all data from all sources, the joint PDF of the observations is

$$p\left(d|\text{Θ}\right)=\prod _{i=1}^N\prod _{k=1}^{N_i}p\left(d_{i,k}|{\mathrm{\theta }}_i\right)$$

Formally the ML estimate is defined as the solution to the following optimization problem:

$$\widehat{\text{Θ}}=\underset{\text{Θ}\in \text{Ω}}{\text{argmin}}\sum _{i=1}^N\sum _{k=1}^{N_i}-\ln p\left(d_{i,k}|{\mathrm{\theta }}_i\right)(3)$$

where Ω accounts for any constraints on the parameters such as the positivity of the σ_c. The solution to (3) was obtained using the optimize function in scikit-learn (Pedregosa et al., 2011). The use of this routine requires the specification of an initial estimate for the parameters in Θ. For b_i and a_i we used the sample standard deviation and sample mean of the i-th data set. With the a_i and b_i as above, we linearly spaced six Gaussians of identical width and height in the interval [−2, 4]. Concretely, the {α_c, μ_c, σ_c} parameters were initialized to

$$\begin{array}{c}{\mathrm{\alpha }}_c={\displaystyle \frac{1}{N_c}}{\mathrm{\sigma }}_c={\displaystyle \frac{1}{N_c}}{\mathrm{\mu }}_c=-2+\left(c-1\right)*{\displaystyle \frac{(4--2)}{N_c-1}}\hfill \\ =-2+{\displaystyle \frac{6(c-1)}{N_c-1}}\hfill \end{array}$$

This approach to initialization allows for a natural interpretation of the linear transformation as generalized Z-score coefficients. The initialization process described above treats each data set independently and takes as given the linear transformed data as the basis for fitting the GMM parameters. The maximum likelihood approach requires jointly optimizes all parameters across the model allowing for the transformation parameters from one data set to influence those of the other data sets as well as the GMM parameters.

Gathering Model Inputs: Joint Distributions

For each pairwise combination of predictor and output we estimated correlation coefficients to define their conditional linear relationships. To develop estimates we gathered effect size estimates from original research and meta-analyses. We prioritized meta-analyses when they existed for a particular pairwise relationship, such as stress effects on executive function (Shields et al., 2016).

In some cases, however, meta-analyses did not exist, and we needed to aggregate effect size estimates from a review of original research articles. For example, no meta-analysis exists relating stress and perceptuo-motor control, so we aggregated across original research studies examining this relationship (Arora et al., 2010b; Bajunaid et al., 2017).

Various effect size measures, such as Cohen’s d and Hedges G, were converted into correlation coefficients (r) to derive a common effect size estimate for each of the nine pairwise combinations (3 × 3). The final effect size matrix relating the three predictors and outcomes is detailed in Table 3, and the original sources for deriving effect sizes are detailed in Table 1. Note that a negative-going coefficient indicates a negative association between variables; for example, the negative coefficient relating physical exertion to reaction time indicates that higher physical exertion levels are generally associated with lower (faster) reaction times. In contrast, the positive coefficient relating physical exertion with executive function errors indicates that higher physical exertion levels are generally associated with higher (more error-prone) executive function task performance.

TABLE 3

Table 3. Effect size estimates (and 95% confidence intervals) relating each of the predictors (stress, sleep, and physical exertion) to each of the outcomes of interest (reaction time, executive function, and perceptuo-motor control).

Adaptive LASSO Model Optimization for Determining the Joint Density

The final component of the model we propose is p(y_i, x₁, x₂, x₃), the joint distribution of an output variable and all three inputs. In this section, we detail a construction of this PDF as the solution to an optimization problem. Specifically, the four-dimensional PDF is represented as a convex superposition of isotropic Gaussian densities of varying widths uniformly distributed in z = (y_i, x₁, x₂, x₃). We choose the coefficients in this expansion such that:

• The marginal distributions for p(y) and p(x_i) computed using the joint model match in an appropriate sense those constructed from the data as detailed in section “Gathering Model Inputs: Marginal Distributions” above. Figure 1 depicts the supplied and recovered marginal distributions for p(y) and p(x_i), also detailed in Table 4.

• The correlation coefficient computed using the joint model similarly, match those obtained from literature in Table 2.

• The LASSO (Tibshirani, 1996) approach resulting in a model which is parsimonious in that only a few coefficients in the expansion are nonzero corresponding to what, in a sense we can make precise, are the “most important” Gaussians.

FIGURE 1

Figure 1. Comparison of true (supplied) and recovered marginal distributions for each of the three dependent (top) and independent (bottom) variables, along with the centers of the Gaussian kernels, which were computed using the LASSO method detailed in section “Gathering Model Inputs: Marginal Distributions”. Note, the marginals for the dependent variables were generated after separate runs of the fitting algorithm using all three independent variables. The marginal distributions shown for the independent variables were obtained from the fit using reaction time (the marginal fits of the independent variables using the other two dependent variables displayed an equal level of accuracy). In addition to the fidelity between the LASSO model (recovered) and data-generated (supplied) marginals, we also observe that the KDE and ML methods for recovering marginals were able to resolve a variety of detail and non-Gaussian structure within the distributions.

TABLE 4

Table 4. Supplied and recovered effect size estimates (r) for each of the three independent (rows) and dependent (columns) variables.

We chose to employ a LASSO-based approach because of its strong theoretical guarantees. Specifically, the LASSO formulation yields a convex optimization problem to determine the α coefficients. Such problems possess a single, globally optimum solution which can be found regardless of how the parameters are initialized in the search process. This fact along with the robust codes that exist for solving the large-scale instances of such problems as arise in this work make the LASSO alternative quite convenient.

To simplify the notation, in this section, we shall drop the i subscript in y_i identifying the specific output variable of interest, as the process is the same for all three outcome variables, and we let x = (x₁, x₂, x₃) be the vector of three predictor variables.

While the marginals obtained via the method in section “Gathering Model Inputs: Marginal Distributions” have meaningful units attached to the associated variables, for the purposes of the joint distribution reconstruction we normalize the variable axis of the marginal to fit within the region [−1, 1]. Formally, we specify a lattice of points consisting of N equally spaced points (with N an odd number) between −1 and 1 on each axis. This will produce a grid of N⁴ points in the 4D space. Letting μ_n ∈ ℝ⁴ denote the n-th point in this 4D grid, at each of these points we place M Gaussians of the form

$$f_{n,m}\left(z\right)=\frac{1}{{\left(2\mathrm{\pi }\right)}^2{\mathrm{\sigma }}_m^4}\exp \left(-\frac{1}{2{\mathrm{\sigma }}_m^2}{||z-{\mathrm{\mu }}_n||}_2^2\right)$$

where the M values for σ_m must be chosen judiciously to allow for a range of “features” in the resulting 4D PDF. For the work here, we found that the following M = 7 values were sufficient:

$${\mathrm{\sigma }}_m\in \{0.03,0.05,0.10,0.15,0.20,0.30,0.40\}.$$

Rather than using two indices, m = 1, 2, …, M and n = 1, 2, …, N⁴, we shall use one index i = 1, 2, …, MN⁴ ≡ N_B so that the above equation now takes the form:

$$f_i\left(z\right)=\frac{1}{{\left(2\mathrm{\pi }\right)}^2{\mathrm{\sigma }}_i^4}\exp \left(-\frac{1}{2{\mathrm{\sigma }}_i^2}{||z-{\mathrm{\mu }}_i||}_2^2\right).$$

The underling ordering of the points can be arbitrary, as long as it is used consistently in the software implementation of this method.

Using these Gaussian basis functions, we represent p(y,x)as a mixture model:

$$p(y,x)=\sum _i{\mathrm{\alpha }}_i{f}_i(y,x)(4)$$

where the expansion coefficients satisfy the probability simplex constraints:

$$0\le {\mathrm{\alpha }}_i\le 1\mathrm{and}\sum _{i=1}^{N_B}{\mathrm{\alpha }}_i=1.(5)$$

Using this model, the marginals are expressed in terms of the α_i coefficients as

$$p\left(y\right)=\sum _i{\mathrm{\alpha }}_i{f}_i\left(y\right)(6)$$

$$p\left(x_j\right)=\sum _i{\mathrm{\alpha }}_i{f}_i\left(x_j\right)j=1,2,3(7)$$

Because f_i(z) is an isotropic Gaussian PDF, f_i(y) in (6) is a univariate Gaussian with mean equal to the first (that is the “y”-th) element of the vector μ_i and standard deviation σ_i. A similar interpretation holds for f_i(x_j). To represent the correlation coefficients in terms of the α_i coefficients, recall that

$$r_{x_{j}y}=\frac{E\left[x_{j}y\right]-E\left[x_j\right]E[y]}{s_{x_j}{s}_y}(8)$$

where s_xj is the standard deviation of x_j and similarly, for s_y. In (8), the means and the standard deviations for the individual random variables are determined from the marginals computed in section “Gathering Model Inputs: Marginal Distributions”. Using (4) in (8) allow us to relate the literature-determined effect sizes to the unknown coefficients as

$$r_{x_{j}y}+\frac{m_{x_j}{m}_y}{s_{x_j}{s}_y}=\sum _i{\mathrm{\alpha }}_i\frac{{\mathrm{\mu }}_{x_{j_i}}{\mathrm{\mu }}_{y_i}}{s_{x_j}{s}_y}.(9)$$

To construct a finite dimensional utility function for finding the α coefficients, we uniformly discretize each of the y and x_i axes into K points. Evaluating (6) and (7) at these K points gives 4K linear relationships between the α_i and the marginals. For example

$$p\left(y_k\right)=\sum _i{\mathrm{\alpha }}_i{f}_i\left(y_k\right)=A_{1,k}\mathrm{\alpha }(10)$$

with A_1,k a row vector containing f_i(y_k) and α the column vector of the N_B α parameters. Collecting all K of these relations gives a matrix-vector model of the form p₁ = A₁α where p₁ is a vector of length K and A₁ is a matrix with K rows and N_B columns. Similarly, we have p₂ = A₂α, p₃ = A₃α, and p₄ = A₄α for the three sets of constraints of (7). Finally, (9) provides three constraints on the α vector resulting in p₅ = A₅α where p₅ is length three and A₅ has three rows and N_B columns.

A typical LASSO approach for selecting α is known to not provide sparse solutions for problems involving simplex constraints (Li et al., 2020). An iterative procedure overcomes this issue by modifying an adaptive LASSO objective as follows

$${\mathrm{\alpha }}^{{}^{*}}=\underset{\mathrm{\alpha }}{\text{argmin}}\sum _{k=1}^5{w}_k{||p_k-A_k\mathrm{\alpha }||}_2^2+\mathrm{\lambda }{||\text{diag}(c)\mathrm{\alpha }||}_1(11)$$

subject to 0 ≤ α_i ≤ 1 and $\sum _i{\mathrm{\alpha }}_i=1$

where c is a weight vector and diag(c) is the diagonal matrix with the vector c on the diagonal. The approach in Li et al. (2020) solved (11) repeatedly, updating c from one iteration to the next to enforce sparsity in α. The formal process is as follows:

(1) Initialize c to (1, 1, …, 1)^T

(2) Run the optimization problem with the objective in (11), obtaining a α^∗

(3) Update the coefficient weight vector c based on α^∗:

$$c_i=\{\begin{array}{cc}\hfill \frac{1}{{\mathrm{\alpha }}_i^{{}^{*}}}\hfill & \hfill \mathrm{if}{\mathrm{\alpha }}_i>t\hfill \\ \hfill \frac{1}{2{\mathrm{\alpha }}_i^{{}^{*}}}\hfill & \hfill \text{else}\hfill \end{array}$$

(4) Threshold α^∗:

${\mathrm{\alpha }}_i^{{}^{*}}=0$ if ${\mathrm{\alpha }}_i^{{}^{*}}\le t$, else leave the value unchanged.

(5) Count the number of nonzero elements in the thresholded α^∗:

(a) If that number is less than or equal to N_z, stop the procedure. The thresholded α^∗ is the final sparse solution.

(b) If that number is higher than N_z, go back to step 2 using the updated c vector and repeat the process.

We have found that coefficients with high weights are forced almost to 0, and thresholding these “almost-0” coefficients to 0 has little effect on the recovered joint distribution while significantly increasing the sparsity. A threshold of t = 10⁻⁹ retained this property even on a problem with 1.3 million coefficients. We implemented this procedure in Python using the cvxpy package (Diamond and Boyd, 2016).

In (11), the w_k can be used to weight the different constraints. We found it useful to set a high weight (w₅ = 75) on the A₅ term, which only has three rows, and a low weight (w_i = 1 for i = 1, 2, 3, 4) on the other A_i terms, which each have K > 3 rows, to balance the importance of fitting the recovered marginal and fitting the recovered r-values. The parameter λ is chosen to balance the impact of the first term in the cost function, which encourages a model that fits the marginals and moment constraints well, against the second term which seeks a vector of coefficient that is “sparse;” in that it contains a few large entries with the remainder small or zero. In the context of our iterative procedure, λ affects the rate at which coefficients get pulled to 0 after each iteration: when λ is large, more coefficients get zeroed out. This provides the user with flexibility to target a specific range of coefficients. Larger values of lambda will likely decrease the number of iterations needed. However, this can also lead to overshooting, and the procedure ends up zeroing out all but 10 coefficients (which is usually too sparse). We have found that targeting a representation containing less than 400 coefficients with λ = 1e⁻⁵ to be a decent compromise, but we encourage readers more concerned about time to try larger values of λ.

For each of the three dependent variables of interest in this study, i.e., simple reaction time, perceptuo-motor control, and executive function, the joint distributions are functions of four variables; namely the single dependent variable and the three independent variables of sleep, stress, and exertion. To visualize these four-dimensional distributions, in Figure 2 we plot 3D slices of the dependent and pairs of independent variables.

FIGURE 2

Figure 2. Three-dimensional (3D) slices of the dependent (rows) and pairs of independent (columns) variables. In each panel, we plot the 3D iso-contours at levels 1, 10, and 50% of maximum value in yellow, red, and gray, respectively. The yellow “blobs” indicate the areas of highest probability density in the 3D space, with the red and gray volumes providing an indication as to how the probability is spread. We also display in pink/cyan shades the two dimensional distributions of the dependent and each independent variable with strong pink shades indicating regions of high probability density; these colormaps were saturated at a value of 10% of maximum value to better display the variability in the distribution functions.

As depicted in Figure 2, the marginal plots provide some insight into the way our approach to constructing the 4D distribution functions performs. It is evident that the Adaptive LASSO approach yields high dimensional distributions which, in all cases, faithfully reproduce the six marginals (Table 4). Moreover, the correlation structure of pairs of variables is also evident in the 2D projections; for example, the image on the second row and first column of Figure 2 involving perceptuo-motor control and exertion. The strong diagonal structure seen in the exertion-perceptuo-motor control plane reflects the 0.578 effect size (Table 3) relating these two variables. Analogously, the much smaller effect size of −0.189 between perceptuo-motor and sleep is seen in the other 2D component of this plot, where the directionally is far less pronounced.

Software Tool Implementation

The described model is realized in a set of Python-based software tools that users can rely upon to understand and interact with the model in an intuitive manner. The first iteration of this software tool and graphical user interface is depicted in Figure 3.

FIGURE 3

Figure 3. Graphical user interface for a software tool allowing users to interact with computational model, depicting three inputs (x variables) and outputs (y variables). Shading surrounding output estimates indicates 95% confidence interval around estimated output.

The interface provides a means for users to modify each input (x) by way of user-inputs (e.g., sliders), and visualize how each predicted output (y) is affected. Because the conditional distributions of y provide complete probabilistic models for each outcome, it is possible to plot confidence intervals around estimates and visualize outcome distributions. Each of these features is depicted in the software tool, with 95% confidence intervals surrounding each estimated model output, and the distribution visualized as a violin plot alongside each outcome variable.

We envision four primary uses for the model and software tool. First, for scientists and engineers to identify gaps in scientific knowledge and plan research endeavors; for example, when characterizing the relationship between stress (x) and perceptuo-motor skill (y), we found very few studies experimentally examining this relationship (with a total sample size of only 41 in total), suggesting value in continuing research in this area. As additional predictors and outcomes are integrated into the model, it is likely that many knowledge gaps will become apparent and motivate research agendas.

Second, we envision the model and software tool to be useful for military decision-makers who are planning future operations in the context of limited personnel resources. For example, a commander might preferentially allocate particular units to various tasks that are more or less physically and/or cognitively demanding [commonly termed force management decisions (Wohl, 1981)]. Insights might be made in the context of real-time data feeds from wearable devices and/or may result from commanders’ intuitions about current or projected demands. In addition to force allocation, insights from the tool might be used to prioritize available resources such as food or water and to introduce ample recovery periods (e.g., rest, sleep) following especially intensive activities.

Third, outcomes of our predictive modeling can help prioritize which internal states and contextual factors are worth targeting with environmental sensors (e.g., ambient temperature, noise, and pollution) and wearable biosensors. For example, if mental stress is associated with substantial performance declines across cognitive or physical domains deemed critical for current or projected operations, it may prove particularly advantageous to develop, procure, and/or deploy wearable biosensors to monitor stress states. Recent progress with biosensors placed in the mouth, such as affixed to a tooth, show promise for sensing salivary analytes linked to stress, such as alpha-amylase and cortisol (Robles et al., 2011; Coppedè et al., 2014; Tseng et al., 2018; Ahn et al., 2019).

Finally, outcomes of our predictive model could be used to inform investments into future performance optimization or enhancement techniques and technologies (Bostrom and Sandberg, 2009; Colzato, 2018; Feltman et al., 2019; Brunyé et al., 2020). For example, if physical encumbrance and exertion is particularly influential for sustained performance across domains of interest, it may warrant increased investment in strength and endurance training, lower-extremity exoskeletons, or other approaches for reducing physical and physiological burden of continuous operations (Gregorczyk et al., 2010; Ozaki et al., 2013; Scribbans et al., 2016; Seo et al., 2018; Wei et al., 2020; Shepertycky et al., 2021). The same might be said for sleep or stress, identifying methods for increasing the quality or duration of naps and overnight sleep (Irwin et al., 2008; Ketz et al., 2018; Robinson et al., 2018), and/or reducing the intensity of stress responses (Stanley et al., 2011; Jaremko and Meichenbaum, 2013). As additional predictors are incorporated into the model, it will afford a more robust ranking and prioritization of cognitive and/or physical states that are most negatively impactful for performance and motivate research and development toward mitigating such impacts.

Limitations and Future Directions

We detailed a preliminary version of a computational model and software tool to enable actionable insights into cognitive and physical performance outcomes by interpreting biosensor data related to sleep status, stress state, and levels of physical workload; given the preliminary nature of the work, there are many directions for continuing research and development.

In Figure 2, we observe distinct spherical structures in these plots in upper left and center right plots. These features arise from our use of isotropic Gaussians of seven fixed widths yielding spherical clusters of similar sizes. Specifically, the LASSO-based scheme chooses a representation of the joint probability density function from a large but finite set of isotropic Gaussians of varying widths on a grid of points in 4D space. The optimization process by which this set is chosen occasionally selects spatially isolated basis functions to help represent small scale features in the marginal distributions. As a result, we see those isolated spherical (since the Gaussian are isotropic) artifacts. To remove these artifacts, our current efforts are aimed at replacing the LASSO scheme with a more flexible method capable of employing Gaussian basis functions with arbitrary center locations and covariance matrices. The cost of such flexibility is an increase in the complexity of the optimization method required to fit the model from data. For example, as the problem is no longer convex in the unknown parameters, we must contend with the challenge of local minima producing poor models. The convex nature of the LASSO scheme used in this manuscript ensures that the model we compute is in fact unique.

The model we developed is ostensibly a main effects model, informed only by effect size coefficients linking single input variables to each output variable. In this manner, no interaction terms are explicitly defined in the preliminary model. However, there could be value in incorporating known interaction terms among our x variables, either of the two- or three-way variety. Reproducibility will be important for incorporating interaction terms into the model. In our review of the extant literature relating sleep, stress, and physical exertion to each of our outcome variables, there were very few studies examining interactions among our input variables. Even when those studies did exist, one study might suggest the presence of a two-way interaction between two of our x variables when predicting reaction time, and another study might show an opposing pattern or no interaction at all. In studies examining sleep deprivation, physical exertion, and reaction time, some reliable interactions have been found. In one study, 30 h of sleep deprivation caused increased reaction times, but this effect was entirely mitigated by brief bouts of physical exertion (Scott et al., 2006). Two additional studies showed that brief morning exercise routines were sufficient to mitigate reaction time costs of partial sleep deprivation in elite athletes (Taheri and Irandoust, 2020), and that acute submaximal physical exertion counteracted reaction time deficits following a single night of sleep deprivation (Temesi et al., 2013). Continuing model development will begin to incorporate interaction terms to better specify relationships among multiple x variables in predicting outcomes. As we pursue the next expanded version of our model, we will be seeking out and incorporating any and all interaction terms that we can identify in the extant literature, and training the model on these patterns. It could be the case that some of these new interactions may qualify or possibly limit the value of the main effects model; it could also be the case that they serve to motivate continuing research to fill knowledge gaps in the literature.

In addition to interaction terms among x variables when predicting outcomes, there also likely exist relationships (i.e., collinearity) among both x (i.e., x-to-x relationships) and y (i.e., y-to-y relationships) variables. For example, when individuals are deprived of sleep they also tend to show higher cortisol levels throughout the day and a potentiated HPA axis response to acute stress exposure (Minkel et al., 2014). Sleep loss can also cause bouts of physical exertion to be experienced as more intense than usual, increasing ratings of perceived exertion (Martin and Gaddis, 1981; Myles, 1985) and reducing time to volitional exhaustion (Van Helder and Radomski, 1989). In other words, sleep loss is associated with increased stress levels (Nollet et al., 2020) and altered behavioral responses to exercise. As it is based only on the marginals, construction of the preliminary model does not enforce any specific interactions among the sleep, stress, and physical exertion; however, these patterns will be important to incorporate into continuing model development. The same can be noted for our outcomes of interest; we realize that some y-to-y relationships may be important to characterize and account for in the construction of the model. For example, slow-downs in reaction time are often accompanied by increases in executive function errors (Willoughby et al., 2020). Populating a complete matrix that expands upon the one detailed in Table 3 will allow us to begin training the model on these x-to-x and y-to-y relationships.

Incorporation of the interactions described in the previous two paragraphs into our probabilistic models can easily be accomplished depending on the nature of the data provided. For example, from jointly collected observations of two input variables, say x₁, and x₂ along with one output y, density estimation methods can produce the corresponding PDF p(y, x₁, x₂) which will provide another A matrix to be fit in the LASSO process. Analogous methods would be used if we were provided data from multiple output and inputs or just multiple inputs.

Whereas time is an inherent property of our sleep variable, it is not incorporated into any other aspect of the model. However, we also know that the influence of some x variables will be time-dependent. For example, extended bouts of moderate- to high-intensity physical exertion are likely more impactful than relatively brief bouts. Indeed meta-analytic investigations of the relationship between physical exertion, duration, and cognitive performance indicate that 1–10 min of exercise carries an effect size of 0.06 with cognitive task performance, whereas 11–20 min shows an effect size of −0.18, and greater than 20 min has an effect size of 0.26 (Chang et al., 2012). Notice how the effect sizes characterizing the relationship between physical exertion and cognition change in magnitude and directionality. In other words, the relationship between physical exertion and cognitive performance may interact with duration in a non-linear manner. There are at least two ways for the model to incorporate such patterns. First, we could develop a new predictor that combines physical exertion and duration into a single exposure variable that replaces the original physical exertion variable; this approach would be dependent on original data sources to help characterize the relationship between exertion- and duration-contingent effects on cognitive outcomes. Second, we could develop multiple marginal input distributions for physical exertion, one each for relatively brief, moderate, and long-duration periods of exertion. Similar approaches may need to be investigated for extended durations of stress, which may shift stress responses from acute to chronic and alter the relationship with our outcomes of interest.

The three predictors and outcomes of interest are by no means an exhaustive representation of the myriad individual (e.g., traits, skills, and experience) and contextual (e.g., thermal load, hydration, and group dynamics) factors, and cognitive and physical performance outcomes characterizing real-world performance. As we continue to develop the model, we will begin to incorporate additional predictors and outcomes that help describe the individuals in a sub-population, and more comprehensively describe predicted outcomes. For outcomes, near-term goals are to incorporate three additional outcomes of interest: gross motor ability (e.g., strength, endurance), memory (e.g., recognition, recall), and communications (e.g., verbal language production and comprehension).

While the present model and tool are designed to ingest and interpret data from wearable devices, either in real-time or with opportunistic sampling, we have not yet integrated sensing and interpretation. With some input variables, such as sleep, there are near-term opportunities for such integration. For example, wearable sleep trackers are becoming common on the consumer market, with options providing daily tracking of sleep quantity and quality. Using either discrete manual updates by users or integrating with a device’s application programming interface (API), the model could receive daily updates to feed the sleep input parameter. Over the next several months, we will be exploring integration of sensor outputs into our model, beginning with sleep and continuing to incorporate physical exertion and stress sensor feeds. While this research topic is specifically related to wearable sensing technologies, it should also be pointed out that many non-contact in-situ sensing technologies are emerging on the market. For example, non-contact stress detection is possible through infrared and microwave doppler radar (Zakrzewski et al., 2012; Lee et al., 2015; Shan et al., 2020), and doppler radar may be used to non-contact measurements of the orientation (and possibly quality) of sleep (Kiriazi et al., 2021). It is possible that non-contact biosensors may meet or exceed the sensitivity and reliability of wearable sensors in the future, may prove valuable complements to wearable sensors, and may increase the breadth and specificity of measurements that could be used in predictive models. One advantage of our approach to modeling several input data types is that we are relatively sensor-agnostic, which ultimately should increase the flexibility of our model to accept novel non-contact sensor inputs.

The current version of the model is trained on both laboratory and field study data, but model predictions have not yet been validated in either setting. We are currently collecting both laboratory and field data in studies intentionally designed to manipulate and/or measure sleep, physical exertion, and mental stress. In these studies, we are measuring a diverse number of cognitive and physical task outcomes that are directly applicable to the output variables of our model. Two specific studies are worth detailing. First, we are conducting an immersive virtual reality study that elicits high levels of mental stress via threat of aversive torso shock (versus a low stress control condition). In this study, we measure performance outcomes including reaction time, recognition memory, executive function, decision making, and perceptuo-motor control (marksmanship aiming). Outcomes from this first study will help validate predictions linking mental stress to the three model outputs, while also providing helpful data to begin expanding the breadth model predictions. Second, we are conducting an exercise physiology study that elicits high levels of physical exertion via a prolonged (2-hour) inclined treadmill walk while carrying a heavy backpack load (versus a no-load control condition). In this study, we measure performance outcomes including reaction time, recognition memory, executive function, decision making, and gross motor output. Outcomes from this second study will help validate predictions linking physical exertion to the three model outputs, additionally providing preliminary data to expand the breadth of model predictions. Due to the SARS-CoV-2 pandemic and associated human subjects research restrictions, these studies have been delayed; we anticipate completing these studies and model validation over the next year. As we continue to collect data in these studies, we will use a portion of these data to continue populating and expanding our model, and a portion to validate model predictions. Based on the fit of model predictions to our data, we will iteratively expand upon training data and/or continue to increase the breadth of the model inputs and outputs.

Finally, the current version of the model and software tool does not fully bridge the gap between our predictors of interest (stress, sleep, and physical exertion) and actionable intelligence for military applications. Whereas scientists and engineers may be interested in predicting such aspects of performance as reaction time and executive function errors, and some trainers and commanders may understand and realize how to use these predictions in their own planning and operations, these specific processes and measures may not be comprehensible or useful for all users. However, we argue that any outcome of interest in an applied setting, such as marksmanship or threat detection ability, can be summarized from weighted combinations of model outputs. For example, task analyses of marksmanship ability demonstrate that aspects of gross and fine motor ability, perceptuo-motor control, and executive function are involved (Chung et al., 2004). Weighted combinations of these output variables can give rise to indirect predictions of marksmanship ability through an understanding of its component processes. This methodology will likely prove valuable when translating model predictions to user-specific needs across a range of applications.

Conclusion

As wearable sensors continue to proliferate the consumer market, they provide opportunities to derive quantitative measures and inform actionable insights into predicted performance outcomes across cognitive and physical domains (Lin et al., 2020). Communicating information to users from wearable devices is the first step in realizing this opportunity, with the second step involving the interpretation and translation of data into predicted outcomes. These predicted outcomes can be used to help plan continuing operations, allocate groups of individuals to tasks with specific demands, identify gaps in knowledge and opportunities for biosensing, and inform enhancement strategies to help mitigate performance-deleterious effects of x variables.

As a first step toward addressing this challenge in the cognitive and physical domain, we detailed an extensible computational model that can translate inputs from wearable actigraphy/sleep sensors, inertial measurement units (IMUs), and/or physiological stress biosensors to actionable insights for leaders and trainers. While preliminary in nature, our model provides a strong foundation for continuing development of human performance predictive capabilities in the cognitive and physical domains.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author.

Author Contributions

TB, EM, and KY drafted the manuscript and revised and expanded the manuscript upon by all other authors. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by the Combat Capabilities Development Command Soldier Center (DEVCOM SC) under the Measuring and Advancing Soldier Tactical Readiness and Effectiveness (MASTR-E) program, via cooperative agreement (W911QY1920003) to the School of Engineering at Tufts University. EM was additionally supported under NSF grants 1934553 and 1931978.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ For the remainder of this manuscript, we let y₁ = reaction time, y₂ = executive function, y₃ = perceptuo − motor control, x₁ = stress, x₂ = sleep, and x₃ = physical exertion.

References

Acheson, A., Richards, J. B., and de Wit, H. (2007). Effects of sleep deprivation on impulsive behaviors in men and women. Physiol. Behav. 91, 579–587. doi: 10.1016/j.physbeh.2007.03.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Ahn, J. W., Ku, Y., and Kim, H. C. (2019). A novel wearable EEG and ECG recording system for stress assessment. Sensors 19:1991. doi: 10.3390/s19091991

PubMed Abstract | CrossRef Full Text | Google Scholar

Ainsworth, B. E., Haskell, W. L., Herrmann, S. D., Meckes, N., Bassett, D. R., Tudor-Locke, C., et al. (2011). 2011 compendium of physical activities: a second update of codes and MET values. Med. Sci. Sports Exerc. 43, 1575–1581. doi: 10.1249/MSS.0b013e31821ece12

PubMed Abstract | CrossRef Full Text | Google Scholar

Ainsworth, B. E., Haskell, W. L., Leon, A. S., Jacobs, D. R., Montoye, H. J., Sallis, J. F., et al. (1993). Compendium of physical activities: classification of energy costs of human physical activities. Med. Sci. Sports Exerc. 25, 71–80. doi: 10.1249/00005768-199301000-00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Ainsworth, B. E., Haskell, W. L., Whitt, M. C., Irwin, M. L., Swartz, A. M., Strath, S. J., et al. (2000). Compendium of physical activities: an update of activity codes and MET intensities. Med. Sci. Sports Exerc. 32(Suppl. 9), S498–S504. doi: 10.1097/00005768-200009001-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

Angelidis, A., Solis, E., Lautenbach, F., van der Does, W., and Putman, P. (2019). I’m going to fail! Acute cognitive performance anxiety increases threat-interference and impairs WM performance. PLoS One 14:e0210824. doi: 10.1371/journal.pone.0210824

PubMed Abstract | CrossRef Full Text | Google Scholar

Angelova, M., Holloway, P. M., Shelyag, S., Rajasegarar, S., and Rauch, H. G. L. (2021). Effect of stress on cardiorespiratory synchronization of ironman athletes. Front. Physiol. 12:612245. doi: 10.3389/fphys.2021.612245

PubMed Abstract | CrossRef Full Text | Google Scholar

Arnsten, A. F. T. (1998). Catecholamine modulation of prefrontal cortical cognitive function. Trends Cogn. Sci. 2, 436–447. doi: 10.1016/S1364-6613(98)01240-6

CrossRef Full Text | Google Scholar

Arnsten, A. F. T. (2009). Stress signalling pathways that impair prefrontal cortex structure and function. Nat. Rev. Neurosci. 10, 410–422. doi: 10.1038/nrn2648

PubMed Abstract | CrossRef Full Text | Google Scholar

Arora, S., Sevdalis, N., Aggarwal, R., Sirimanna, P., Darzi, A., and Kneebone, R. (2010a). Stress impairs psychomotor performance in novice laparoscopic surgeons. Surg. Endosc. 24, 2588–2593. doi: 10.1007/s00464-010-1013-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Arora, S., Sevdalis, N., Nestel, D., Woloshynowych, M., Darzi, A., and Kneebone, R. (2010b). The impact of stress on surgical performance: a systematic review of the literature. Surgery 147, 318–330.e6. doi: 10.1016/j.surg.2009.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Axelrod, J., and Reisine, T. D. (1984). Stress hormones: their interaction and regulation. Science 224, 452–459. doi: 10.1126/science.6143403

PubMed Abstract | CrossRef Full Text | Google Scholar

Bajunaid, K., Mullah, M. A. S., Winkler-Schwartz, A., Alotaibi, F. E., Fares, J., Baggiani, M., et al. (2017). Impact of acute stress on psychomotor bimanual performance during a simulated tumor resection task. J. Neurosurg. 126, 71–80. doi: 10.3171/2015.5.JNS15558

PubMed Abstract | CrossRef Full Text | Google Scholar

Barrett, B., Cruickshank, A., Flavell, J., Bennett, S., Buckley, J., Harris, J., et al. (2020). Faster visual reaction times in elite athletes are not linked to better gaze stability. Sci. Rep. 10:13216. doi: 10.1038/s41598-020-69975-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Bartel, P., Offermeier, W., Smith, F., and Becker, P. (2004). Attention and working memory in resident anaesthetists after night duty: group and individual effects. Occup. Environ. Med. 61, 167–170. doi: 10.1136/oem.2002.006197

PubMed Abstract | CrossRef Full Text | Google Scholar

Baumert, A., Buchholz, N., Zinkernagel, A., Clarke, P., MacLeod, C., Osinsky, R., et al. (2020). Causal underpinnings of working memory and Stroop interference control: testing the effects of anodal and cathodal tDCS over the left DLPFC. Cogn. Affect. Behav. Neurosci. 20, 34–48. doi: 10.3758/s13415-019-00726-y

PubMed Abstract | CrossRef Full Text | Google Scholar

Besedovsky, L., Lange, T., and Born, J. (2012). Sleep and immune function. Pflügers Arch. Eur. J. Physiol. 463, 121–137. doi: 10.1007/s00424-011-1044-0

PubMed Abstract | CrossRef Full Text | Google Scholar

Biggs, A. T. (2021). Developing scenarios that evoke shoot/don’t-shoot errors. Appl. Ergon. 94:103397. doi: 10.1016/j.apergo.2021.103397

PubMed Abstract | CrossRef Full Text | Google Scholar

Biggs, A. T., Cain, M. S., and Mitroff, S. R. (2015). Cognitive training can reduce civilian casualties in a simulated shooting environment. Psychol. Sci. 26, 1164–1176. doi: 10.1177/0956797615579274

PubMed Abstract | CrossRef Full Text | Google Scholar

Bock, O., Haeger, M., and Voelcker-Rehage, C. (2019). Structure of executive functions in young and in older persons. PLoS One 14:e0216149. doi: 10.1371/journal.pone.0216149

PubMed Abstract | CrossRef Full Text | Google Scholar

Bostrom, N., and Sandberg, A. (2009). Cognitive enhancement: methods, ethics, regulatory challenges. Sci. Eng. Ethics 15, 311–341. doi: 10.1007/s11948-009-9142-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Boucsein, W., Fowles, D. C., Grimnes, S., Ben-Shakhar, G., Roth, W. T., Dawson, M. E., et al. (2012). Publication recommendations for electrodermal measurements. Psychophysiology 49, 1017–1034. doi: 10.1111/j.1469-8986.2012.01384.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Brisswalter, J., Arcelin, R., Audiffren, M., and Delignières, D. (1997). Influence of physical exercise on simple reaction time: effect of physical fitness. Percept. Motor Skills 85(3 Pt 1), 1019–1027. doi: 10.2466/pms.1997.85.3.1019

PubMed Abstract | CrossRef Full Text | Google Scholar

Brisswalter, J., Durand, M., Delignieres, D., and Legros, P. (1995). Optimal and non-optimal demand in a dual task of pedalling and simple reaction time: effects on energy expenditure and cognitive performance. J. Hum. Mov. Stud. 29, 15–34.

Google Scholar

Broadbent, D. E. (1971). Decision and stress. London: Academic Press, xiv, 522.

Google Scholar

Brunyé, T. T., Brou, R., Doty, T. J., Gregory, F. D., Hussey, E. K., Lieberman, H. R., et al. (2020). A review of US army research contributing to cognitive enhancement in military contexts. J. Cogn. Enhanc. 4, 453–468. doi: 10.1007/s41465-020-00167-3

CrossRef Full Text | Google Scholar

Brunyé, T. T., Howe, J. L., Walker, L. A., and Mahoney, C. R. (2013). Acute bouts of endurance exercise increase distractibility to emotional stimuli. Int. J. Sport Psychol. 44, 471–492.

Google Scholar

Buchheim, J.-I., Matzel, S., Rykova, M., Vassilieva, G., Ponomarev, S., Nichiporuk, I., et al. (2019). Stress related shift toward inflammaging in cosmonauts after long-duration space flight. Front. Physiol. 10:85. doi: 10.3389/fphys.2019.00085

PubMed Abstract | CrossRef Full Text | Google Scholar

Burle, B., Vidal, F., Tandonnet, C., and Hasbroucq, T. (2004). Physiological evidence for response inhibition in choice reaction time tasks. Brain Cogn. 56, 153–164. doi: 10.1016/j.bandc.2004.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Byrne, N. M., Hills, A. P., Hunter, G. R., Weinsier, R. L., and Schutz, Y. (2005). Metabolic equivalent: one size does not fit all. J. Appl. Physiol. 99, 1112–1119. doi: 10.1152/japplphysiol.00023.2004

PubMed Abstract | CrossRef Full Text | Google Scholar

Cerqueira, J. J., Mailliet, F., Almeida, O. F. X., Jay, T. M., and Sousa, N. (2007). The prefrontal cortex as a key target of the maladaptive response to stress. J. Neurosci. 27, 2781–2787. doi: 10.1523/JNEUROSCI.4372-06.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Chang, Y.-K., Labban, J. D., Gapin, J. I., and Etnier, J. L. (2012). The effects of acute exercise on cognitive performance: a meta-analysis. Brain Res. 1453, 87–101. doi: 10.1016/j.brainres.2012.02.068

PubMed Abstract | CrossRef Full Text | Google Scholar

Charmandari, E., Tsigos, C., and Chrousos, G. (2005). Endocrinology of the stress response. Annu. Rev. Physiol. 67, 259–284. doi: 10.1146/annurev.physiol.67.040403.120816

PubMed Abstract | CrossRef Full Text | Google Scholar

Chee, M. W. L., and Tan, J. C. (2010). Lapsing when sleep deprived: neural activation characteristics of resistant and vulnerable individuals. NeuroImage 51, 835–843. doi: 10.1016/j.neuroimage.2010.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

Chung, G. K. W. K., De La Cruz, G. C., de Vries, L. F., Kim, J.-O., Bewley, W. L., de Souza e Silva, A., et al. (2004). Determinants of Rifle Marksmanship Performance: Predicting Shooting Performance with Advanced Distributed Learning Assessments (No. ADA453871). Los Angeles, CA: University of California.

Google Scholar

Collardeau, M., Brisswalter, J., and Audiffren, M. (2001). Effects of a prolonged run on simple reaction time of well trained runners. Percept. Motor Skills 93, 679–689. doi: 10.2466/pms.2001.93.3.679

PubMed Abstract | CrossRef Full Text | Google Scholar

Colzato, L. S. (2018). Responsible cognitive enhancement: neuroethical considerations. J. Cogn. Enhanc. 2, 331–334. doi: 10.1007/s41465-018-0090-3

CrossRef Full Text | Google Scholar

Connaboy, C., LaGoy, A. D., Johnson, C. D., Sinnott, A. M., Eagle, S. R., Bower, J. L., et al. (2020). Sleep deprivation impairs affordance perception behavior during an action boundary accuracy assessment. Acta Astronaut. 166, 270–276. doi: 10.1016/j.actaastro.2019.10.029

CrossRef Full Text | Google Scholar

Coppedè, N., Tarabella, G., Villani, M., Calestani, D., Iannotta, S., and Zappettini, A. (2014). Human stress monitoring through an organic cotton-fiber biosensor. J. Mater. Chem. B 2, 5620–5626. doi: 10.1039/C4TB00317A

PubMed Abstract | CrossRef Full Text | Google Scholar

Curtis, C. E., and D’Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends Cogn. Sci. 7, 415–423. doi: 10.1016/S1364-6613(03)00197-9

CrossRef Full Text | Google Scholar

Danah, B., and Crawford, K. (2012). Critical questions for big data. Inf. Commun. Soc. 15, 662–679. doi: 10.1080/1369118X.2012.678878

CrossRef Full Text | Google Scholar

Deslandes, A., Ferreira, C., Veiga, H., Cagy, M., Piedade, R., Pompeu, F., et al. (2006). Effects of caffeine on electrophysiological and neuropsychological indices after sleep deprivation. Neuropsychobiology 54, 126–133. doi: 10.1159/000098263

PubMed Abstract | CrossRef Full Text | Google Scholar

Diamond, S., and Boyd, S. (2016). CVXPY: a python-embedded modeling language for convex optimization. J. Mach. Learn. Res. JMLR 17:83.

Google Scholar

Dietrich, A. (2006). Transient hypofrontality as a mechanism for the psychological effects of exercise. Psychiatry Res. 145, 79–83. doi: 10.1016/j.psychres.2005.07.033

PubMed Abstract | CrossRef Full Text | Google Scholar

Dietrich, A., and Audiffren, M. (2011). The reticular-activating hypofrontality (RAH) model of acute exercise. Neurosci. Biobehav. Rev. 35, 1305–1325. doi: 10.1016/j.neubiorev.2011.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Dinges, D. F., Douglas, S. D., Hamarman, S., Zaugg, L., and Kapoor, S. (1995). Sleep deprivation and human immune function. Adv. Neuroimmunol. 5, 97–110. doi: 10.1016/0960-5428(95)00002-j

CrossRef Full Text | Google Scholar

Dominski, F. H., Crocetta, T. B., Santo, L. B., do, E., Cardoso, T. E., da Silva, R., et al. (2018). Police officers who are physically active and have low levels of body fat show better reaction time. J. Occup. Environ. Med. 60, e1–e5. doi: 10.1097/JOM.0000000000001205

PubMed Abstract | CrossRef Full Text | Google Scholar

Donders, F. C. (1969). On the speed of mental processes. Acta Psychol. 30, 412–431. doi: 10.1016/0001-6918(69)90065-1

CrossRef Full Text | Google Scholar

Doran, S. M., Dongen, H. P. A. V., and Dinges, D. F. (2001). Sustained attention performance during sleep deprivation: evidence of state instability. Arch. Ital. Biol. 139, 253–267. doi: 10.4449/aib.v139i3.503

CrossRef Full Text | Google Scholar

Drummond, S. P. A., Paulus, M. P., and Tapert, S. F. (2006). Effects of two nights sleep deprivation and two nights recovery sleep on response inhibition. J. Sleep Res. 15, 261–265. doi: 10.1111/j.1365-2869.2006.00535.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Dunican, I. C., Murray, K., Slater, J. A., Maddison, K. J., Jones, M. J., Dawson, B., et al. (2018). Laboratory and home comparison of wrist-activity monitors and polysomnography in middle-aged adults. Sleep Biol. Rhythms 16, 85–97. doi: 10.1007/s41105-017-0130-x

CrossRef Full Text | Google Scholar

Eddy, M. D. D., Hasselquist, L., Giles, G., Hayes, J. F. J. F., Howe, J., Rourke, J., et al. (2015). The effects of load carriage and physical fatigue on cognitive performance. PLoS One 10:e0130817. doi: 10.1371/journal.pone.0130817

PubMed Abstract | CrossRef Full Text | Google Scholar

Elton, A., and Gao, W. (2014). Divergent task-dependent functional connectivity of executive control and salience networks. Cortex 51, 56–66. doi: 10.1016/j.cortex.2013.10.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Eriksen, B. A., and Eriksen, C. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 16, 143–149. doi: 10.3758/BF03203267

CrossRef Full Text | Google Scholar

Esposito, F., Orizio, C., and Veicsteinas, A. (1998). Electromyogram and mechanomyogram changes in fresh and fatigued muscle during sustained contraction in men. Eur. J. Appl. Physiol. Occup. Physiol. 78, 494–501. doi: 10.1007/s004210050451

PubMed Abstract | CrossRef Full Text | Google Scholar

Etnier, J. L., Salazar, W., Landers, D. M., Petruzzello, S. J., Han, M., and Nowell, P. (1997). The influence of physical fitness and exercise upon cognitive functioning: a meta-analysis. J. Sport Exerc. Psychol. 19, 249–277.

Google Scholar

Falleti, M. G., Maruff, P., Collie, A., Darby, D. G., and McStephen, M. (2003). Qualitative similarities in cognitive impairment associated with 24 h of sustained wakefulness and a blood alcohol concentration of 0.05%. J. Sleep Res. 12, 265–274. doi: 10.1111/j.1365-2869.2003.00363.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Feltman, K. A., Hayes, A. M., Bernhardt, K. A., Nwala, E., and Kelley, A. M. (2019). Viability of tDCS in military environments for performance enhancement: a systematic review. Mil. Med. 185, e53–e60. doi: 10.1093/milmed/usz189

PubMed Abstract | CrossRef Full Text | Google Scholar

Fiedler, K., McCaughey, L., Prager, J., Eichberger, J., and Schnell, K. (2020). Speed-accuracy trade-offs in sample-based decisions. J. Exp. Psychol. doi: 10.1037/xge0000986

PubMed Abstract | CrossRef Full Text | Google Scholar

Forest, G., and Godbout, R. (2000). Effects of sleep deprivation on performance and EEG spectral analysis in young adults. Brain Cogn. 43, 195–200.

Google Scholar

Friedl, K. E. (2018). Military applications of soldier physiological monitoring. J. Sci. Med. Sport 21, 1147–1153. doi: 10.1016/j.jsams.2018.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Gagnon, S. A., and Wagner, A. D. (2016). Acute stress and episodic memory retrieval: neurobiological mechanisms and behavioral consequences. Ann. N. Y. Acad. Sci. 1369, 55–75.

Google Scholar

Gaillard, A. W. K., Soeters, J. M. L. M., and Delahaij, R. (2006). Stress Training and the New Military Environment. In None (EN). Soesterberg: Ministerie van Defensie - NLDA. Available online at: https://surfsharekit.nl/public/d39c59fe-3dee-4876-9a28-c7c12b66617f (accessed February 02, 2021).

Google Scholar

Gallicchio, G., Finkenzeller, T., Sattlecker, G., Lindinger, S., and Hoedlmoser, K. (2019). The influence of physical exercise on the relation between the phase of cardiac cycle and shooting accuracy in biathlon. Eur. J. Sport Sci. 19, 567–575. doi: 10.1080/17461391.2018.1535626

PubMed Abstract | CrossRef Full Text | Google Scholar

Gazzola, V., and Keysers, C. (2009). The observation and execution of actions share motor and somatosensory voxels in all tested subjects: single-subject analyses of unsmoothed fMRI data. Cereb. Cortex 19, 1239–1255. doi: 10.1093/cercor/bhn181

PubMed Abstract | CrossRef Full Text | Google Scholar

Giles, G. E., Mahoney, C. R., Brunyé, T. T., Taylor, H. A., and Kanarek, R. B. (2014). Stress effects on mood, HPA axis, and autonomic response: comparison of three psychosocial stress paradigms. PLoS One 9:e113618. doi: 10.1371/journal.pone.0113618

PubMed Abstract | CrossRef Full Text | Google Scholar

Glenville, M., Broughton, R., Wing, A. M., and Wilkinson, R. T. (1978). Effects of sleep deprivation on short duration performance measures compared to the Wilkinson auditory vigilance task. Sleep 1, 169–176. doi: 10.1093/sleep/1.2.169

PubMed Abstract | CrossRef Full Text | Google Scholar

Goldfarb, E. V., Tompary, A., Davachi, L., and Phelps, E. A. (2019). Acute stress throughout the memory cycle: diverging effects on associative and item memory. J. Exp. Psychol. Gen. 148, 13–29. doi: 10.1037/xge0000472

PubMed Abstract | CrossRef Full Text | Google Scholar

Good, C. H., Brager, A. J., Capaldi, V. F., and Mysliwiec, V. (2020). Sleep in the United States Military. Neuropsychopharmacology 45, 176–191. doi: 10.1038/s41386-019-0431-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Grebot, C., Groslambert, A., Pernin, J.-N., Burtheret, A., and Rouillon, J.-D. (2003). Effects of exercise on perceptual estimation and short-term recall of shooting performance in a biathlon. Percept. Motor Skills 97(3 Pt 2), 1107–1114. doi: 10.2466/pms.2003.97.3f.1107

PubMed Abstract | CrossRef Full Text | Google Scholar

Gregorczyk, K. N., Hasselquist, L., Schiffman, J. M., Bensel, C. K., Obusek, J. P., and Gutekunst, D. J. (2010). Effects of a lower-body exoskeleton device on metabolic cost and gait biomechanics during load carriage. Ergonomics 53, 1263–1275. doi: 10.1080/00140139.2010.512982

PubMed Abstract | CrossRef Full Text | Google Scholar

Groeger, J. A., Viola, A. U., Lo, J. C. Y., von Schantz, M., Archer, S. N., and Dijk, D.-J. (2008). Early morning executive functioning during sleep deprivation is compromised by a PERIOD3 polymorphism. Sleep 31, 1159–1167.

Google Scholar

Harris, D. J., Vine, S. J., Wilson, M. R., McGrath, J. S., LeBel, M.-E., and Buckingham, G. (2017). The effect of observing novice and expert performance on acquisition of surgical skills on a robotic platform. PLoS One 12:e0188233. doi: 10.1371/journal.pone.0188233

PubMed Abstract | CrossRef Full Text | Google Scholar

Heuberger, J. A. A. C., Gal, P., Stuurman, F. E., de Muinck Keizer, W. A. S., Mejia Miranda, Y., and Cohen, A. F. (2018). Repeatability and predictive value of lactate threshold concepts in endurance sports. PLoS One 13:e0206846. doi: 10.1371/journal.pone.0206846

PubMed Abstract | CrossRef Full Text | Google Scholar

Hogervorst, E., Riedel, W., Jeukendrup, A., and Jolles, J. (1996). Cognitive performance after strenuous physical exercise. Percept. Motor Skills 83, 479–488. doi: 10.2466/pms.1996.83.2.479

PubMed Abstract | CrossRef Full Text | Google Scholar

Holden, J. G., Flach, J. M., and Donchin, Y. (1999). Perceptual-motor coordination in an endoscopic surgery simulation. Surg. Endosc. 13, 127–132. doi: 10.1007/s004649900920

PubMed Abstract | CrossRef Full Text | Google Scholar

Holgado, D., Troya, E., Perales, J. C., Vadillo, M. A., and Sanabria, D. (2020). Does mental fatigue impair physical performance? A replication study. Eur. J. Sport Sci. 21, 762–770. doi: 10.1080/17461391.2020.1781265

PubMed Abstract | CrossRef Full Text | Google Scholar

Howell, N., Devendorf, L., Vega Gálvez, T. A., Tian, R., and Ryokai, K. (2018). “Tensions of data-driven reflection: a case study of real-time emotional biosensing,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, (New York, NY: ACM), 1–13. doi: 10.1145/3173574.3174005

CrossRef Full Text | Google Scholar

Hutchinson, M. J., Paulson, T. A. W., Eston, R., and Goosey-Tolfrey, V. L. (2017). Assessment of peak oxygen uptake during handcycling: test-retest reliability and comparison of a ramp-incremented and perceptually-regulated exercise test. PLoS One 12:e0181008. doi: 10.1371/journal.pone.0181008

PubMed Abstract | CrossRef Full Text | Google Scholar

Inoue, A., Impellizzeri, F. M., Pires, F. O., Pompeu, F. A. M. S., Deslandes, A. C., and Santos, T. M. (2016). Effects of sprint versus high-intensity aerobic interval training on cross-country mountain biking performance: a randomized controlled trial. PLoS One 11:e0145298. doi: 10.1371/journal.pone.0145298

PubMed Abstract | CrossRef Full Text | Google Scholar

Irwin, M. R., Olmstead, R., and Motivala, S. J. (2008). Improving sleep quality in older adults with moderate sleep complaints: a randomized controlled trial of tai chi chih. Sleep 31, 1001–1008. doi: 10.5665/sleep/31.7.1001

CrossRef Full Text | Google Scholar

Izenman, A. J. (1991). Review papers: recent developments in nonparametric density estimation. J. Am. Stat. Assoc. 86, 205–224. doi: 10.1080/01621459.1991.10475021

CrossRef Full Text | Google Scholar

Jackson, M. L., Croft, R. J., Kennedy, G. A., Owens, K., and Howard, M. E. (2013). Cognitive components of simulated driving performance: sleep loss effects and predictors. Accid. Anal. Prev. 50, 438–444. doi: 10.1016/j.aap.2012.05.020

PubMed Abstract | CrossRef Full Text | Google Scholar

Jaremko, M., and Meichenbaum, D. (2013). Stress Reduction and Prevention. Berlin: Springer Science & Business Media.

Google Scholar

Johnson, D., Stepan, M., Cesario, J., and Fenn, K. (2020). Sleep deprivation and racial bias in the decision to shoot: a diffusion model analysis. Soc. Psychol. Pers. Sci. 12, 638–647. doi: 10.1177/1948550620932723

CrossRef Full Text | Google Scholar

Ketz, N., Jones, A. P., Bryant, N. B., Clark, V. P., and Pilly, P. K. (2018). Closed-Loop slow-wave tACS improves sleep-dependent long-term memory generalization by modulating endogenous oscillations. J. Neurosci. 38, 7314–7326. doi: 10.1523/JNEUROSCI.0273-18.2018

PubMed Abstract | CrossRef Full Text | Google Scholar

Killgore, W. D. S. (2010). Effects of sleep deprivation on cognition. Prog. Brain Res. 185, 105–129. doi: 10.1016/B978-0-444-53702-7.00007-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Killgore, W. D. S., Kahn-Greene, E. T., Grugle, N. L., Killgore, D. B., and Balkin, T. J. (2009). Sustaining executive functions during sleep deprivation: a comparison of caffeine, dextroamphetamine, and modafinil. Sleep 32, 205–216. doi: 10.1093/sleep/32.2.205

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, E. J., Pellman, B., and Kim, J. J. (2015). Stress effects on the hippocampus: a critical review. Learn. Mem. 22, 411–416. doi: 10.1101/lm.037291.114

PubMed Abstract | CrossRef Full Text | Google Scholar

Kiriazi, J. E., Islam, S. M. M., Borić-Lubecke, O., and Lubecke, V. M. (2021). Sleep posture recognition with a dual-frequency cardiopulmonary doppler radar. IEEE Access 9, 36181–36194. doi: 10.1109/ACCESS.2021.3062385

CrossRef Full Text | Google Scholar

Klein, R. J., Liu, T., Diehl, D., and Robinson, M. D. (2017). The personality-related implications of Stroop performance: stress-contingent self-control in daily life. J. Res. Pers. 70, 156–165. doi: 10.1016/j.jrp.2017.07.006

CrossRef Full Text | Google Scholar

Knelange, E. B., and López-Moliner, J. (2020). Increased error-correction leads to both higher levels of variability and adaptation. PLoS One 15:e0227913. doi: 10.1371/journal.pone.0227913

PubMed Abstract | CrossRef Full Text | Google Scholar

Knutson, K. L., Phelan, J., Paskow, M. J., Roach, A., Whiton, K., Langer, G., et al. (2017). The national sleep foundation’s sleep health index. Sleep Health 3, 234–240. doi: 10.1016/j.sleh.2017.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Kosinski, R. A. (2010). A Literature Review on Reaction Time. Available online at: http://www.cognaction.org/cogs105/readings/clemson.rt.pdf (accessed February 02, 2021).

Google Scholar

Kutilek, P., Volf, P., Viteckova, S., Smrcka, P., Krivanek, V., Lhotska, L., et al. (2017). “Wearable systems for monitoring the health condition of soldiers: review and application,” in Proceedings of the 2017 International Conference on Military Technologies (ICMT), (Piscataway, NY: IEEE), 748–752. doi: 10.1109/MILTECHS.2017.7988856

CrossRef Full Text | Google Scholar

Lambourne, K., and Tomporowski, P. (2010). The effect of exercise-induced arousal on cognitive task performance: a meta-regression analysis. Brain Res. 1341, 12–24. doi: 10.1016/j.brainres.2010.03.091

PubMed Abstract | CrossRef Full Text | Google Scholar

Larsen, R. J., Raine, L. B., Hillman, C. H., Kramer, A. F., Cohen, N. J., and Barbey, A. K. (2020). Body mass and cardiorespiratory fitness are associated with altered brain metabolism. Metab. Brain Dis. 35, 999–1007. doi: 10.1007/s11011-020-00560-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Larsson, G. (1989). Personality, appraisal and cognitive coping processes, and performance during various conditions of stress. Mil. Psychol. 1, 167–182. doi: 10.1207/s15327876mp0103_4

CrossRef Full Text | Google Scholar

Lee, H.-J., Kim, L., and Suh, K.-Y. (2003). Cognitive deterioration and changes of P300 during total sleep deprivation. Psychiatry Clin. Neurosci. 57, 490–496. doi: 10.1046/j.1440-1819.2003.01153.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, Y. S., Pathirana, P. N., Evans, R. J., and Steinfort, C. L. (2015). Noncontact detection and analysis of respiratory function using microwave doppler radar. J. Sensors 2015:e548136. doi: 10.1155/2015/548136

CrossRef Full Text | Google Scholar

Li, P., Rangapuram, S. S., and Slawski, M. (2020). METHODS for SPARSE and LOW-RANK RECOVERY under SIMPLEX CONSTRAINTS. Stat. Sin. 30, 557–577. doi: 10.5705/ss.202016.0220

PubMed Abstract | CrossRef Full Text | Google Scholar

Lieberman, H. R., Tharion, W. J., Shukitt-Hale, B., Speckman, K. L., and Tulley, R. (2002). Effects of caffeine, sleep loss, and stress on cognitive performance and mood during U.S. Navy SEAL training. Sea-Air-Land. Psychopharmacology 164, 250–261. doi: 10.1007/s00213-002-1217-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Lim, J., and Dinges, D. F. (2008). Sleep deprivation and vigilant attention. Ann. N. Y. Acad. Sci. 1129, 305–322. doi: 10.1196/annals.1417.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Lim, J., and Dinges, D. F. (2010). A meta-analysis of the impact of short-term sleep deprivation on cognitive variables. Psychol. Bull. 136, 375–389. doi: 10.1037/a0018883

PubMed Abstract | CrossRef Full Text | Google Scholar

Lim, J., Tan, J. C., Parimal, S., Dinges, D. F., and Chee, M. W. L. (2010). Sleep deprivation impairs object-selective attention: a view from the ventral visual cortex. PLoS One 5:e9087. doi: 10.1371/journal.pone.0009087

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, H., Saunders, B., Friese, M., Evans, N. J., and Inzlicht, M. (2020). Strong effort manipulations reduce response caution: a preregistered reinvention of the ego-depletion paradigm. Psychol. Sci. 31, 531–547. doi: 10.1177/0956797620904990

PubMed Abstract | CrossRef Full Text | Google Scholar

Lin, Y., Bariya, M., and Javey, A. (2020). Wearable biosensors for body computing. Adv. Funct. Mater. 2008087, 1–21. doi: 10.1002/adfm.202008087

CrossRef Full Text | Google Scholar

Lipszyc, J., and Schachar, R. (2010). Inhibitory control and psychopathology: a meta-analysis of studies using the stop signal task. J. Int. Neuropsychol. Soc. JINS 16, 1064–1076. doi: 10.1017/S1355617710000895

PubMed Abstract | CrossRef Full Text | Google Scholar

Lisper, H. O., and Kjellberg, A. (1972). Effects of 24-hour sleep deprivation on rate of decrement in a 10-minute auditory reaction time task. J. Exp. Psychol. 96, 287–290. doi: 10.1037/h0033615

PubMed Abstract | CrossRef Full Text | Google Scholar

Liu, Y., and Du, S. (2018). Psychological stress level detection based on electrodermal activity. Behav. Brain Res. 341, 50–53. doi: 10.1016/j.bbr.2017.12.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Logan, G. D. (1985). Executive control of thought and action. Acta Psychol. 60, 193–210. doi: 10.1016/0001-6918(85)90055-1

CrossRef Full Text | Google Scholar

Lou, Z., Wang, L., and Shen, G. (2018). Recent advances in smart wearable sensing systems. Adv. Mater. Tech. 3:1800444. doi: 10.1002/admt.201800444

CrossRef Full Text | Google Scholar

Lou, Z., Wang, L., Jiang, K., Wei, Z., and Shen, G. (2020). Reviews of wearable healthcare systems: materials, devices and system integration. Mater. Sci. Eng. R Rep. 140:100523. doi: 10.1016/j.mser.2019.100523

CrossRef Full Text | Google Scholar

Louw, T., Markkula, G., Boer, E., Madigan, R., Carsten, O., and Merat, N. (2017). Coming back into the loop: drivers’ perceptual-motor performance in critical events after automated driving. Accid. Anal. Prev. 108, 9–18. doi: 10.1016/j.aap.2017.08.011

PubMed Abstract | CrossRef Full Text | Google Scholar

Lowe, C. J., Safati, A., and Hall, P. A. (2017). The neurocognitive consequences of sleep restriction: a meta-analytic review. Neurosci. Biobehav. Rev. 80, 586–604. doi: 10.1016/j.neubiorev.2017.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

Luce, R. D. (1986). Response Times: Their Role in Inferring Elementary Mental Organization. New York, NY: Oxford University Press.

Google Scholar

Ludyga, S., Gerber, M., Brand, S., Holsboer-Trachsler, E., and Pühse, U. (2016). Acute effects of moderate aerobic exercise on specific aspects of executive function in different age and fitness groups: a meta-analysis. Psychophysiology 53, 1611–1626. doi: 10.1111/psyp.12736

PubMed Abstract | CrossRef Full Text | Google Scholar

Luxton, D. D., Greenburg, D., Ryan, J., Niven, A., Wheeler, G., and Mysliwiec, V. (2011). Prevalence and impact of short sleep duration in redeployed OIF soldiers. Sleep 34, 1189–1195. doi: 10.5665/sleep.1236

PubMed Abstract | CrossRef Full Text | Google Scholar

MacLeod, C. M. (2005). “The stroop task in cognitive research,” in Cognitive Methods and Their Application to Clinical Research, eds A. Wenzel and D. C. Rubin (Washington, DC: American Psychological Association), 17–40. doi: 10.1037/10870-002

CrossRef Full Text | Google Scholar

Madore, K. P., Khazenzon, A. M., Backes, C. W., Jiang, J., Uncapher, M. R., Norcia, A. M., et al. (2020). Memory failure predicted by attention lapsing and media multitasking. Nature 587, 87–91. doi: 10.1038/s41586-020-2870-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Martin, B. J., and Gaddis, G. M. (1981). Exercise after sleep deprivation. Med. Sci. Sports Exerc. 13, 220–223. doi: 10.1249/00005768-198104000-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Mathiassen, S. E., Hallman, D. M., Lyskov, E., and Hygge, S. (2014). Can cognitive activities during breaks in repetitive manual work accelerate recovery from fatigue? A controlled experiment. PLoS One 9:e112090. doi: 10.1371/journal.pone.0112090

PubMed Abstract | CrossRef Full Text | Google Scholar

McEwen, B. S. (2007). Physiology and neurobiology of stress and adaptation: central role of the brain. Physiol. Rev. 87, 873–904. doi: 10.1152/physrev.00041.2006

PubMed Abstract | CrossRef Full Text | Google Scholar

McMorris, T., and Keen, P. (1994). Effect of exercise on simple reaction times of recreational athletes. Percept. Motor Skills 78, 123–130. doi: 10.2466/pms.1994.78.1.123

PubMed Abstract | CrossRef Full Text | Google Scholar

McMorris, T., Sproule, J., Turner, A., and Hale, B. J. (2011). Acute, intermediate intensity exercise, and speed and accuracy in working memory tasks: a meta-analytical comparison of effects. Physiol. Behav. 102, 421–428. doi: 10.1016/j.physbeh.2010.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

Medic, G., Wille, M., and Hemels, M. E. (2017). Short- and long-term health consequences of sleep disruption. Nat. Sci. Sleep 9, 151–161. doi: 10.2147/NSS.S134864

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, E. K., and Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202. doi: 10.1146/annurev.neuro.24.1.167

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, E. K., and Wallis, J. D. (2009). “Executive function and higher-order cognition: definition and neural substrates,” in Encyclopedia of Neuroscience, Vol. 4, ed. L. R. Squire (Oxford: Academic Press), 99–104.

Google Scholar

Minkel, J., Moreta, M., Muto, J., Htaik, O., Jones, C., Basner, M., et al. (2014). Sleep deprivation potentiates HPA axis stress reactivity in healthy adults. Health Psychol. 33, 1430–1434. doi: 10.1037/a0034219

PubMed Abstract | CrossRef Full Text | Google Scholar

Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., and Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “Frontal Lobe” tasks: a latent variable analysis. Cogn. Psychol. 41, 49–100. doi: 10.1006/cogp.1999.0734

PubMed Abstract | CrossRef Full Text | Google Scholar

Mokaya, F., Lucas, R., Noh, H. Y., and Zhang, P. (2016). “Burnout: a wearable system for unobtrusive skeletal muscle fatigue estimation,” in Proceedings of the 2016 15th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), (Piscataway, NY: IEEE Press), 1–12. doi: 10.1109/IPSN.2016.7460661

CrossRef Full Text | Google Scholar

Moorthy, K., Munz, Y., Dosis, A., Bann, S., and Darzi, A. (2003). The effect of stress-inducing conditions on the performance of a laparoscopic task. Surg. Endosc. 17, 1481–1484. doi: 10.1007/s00464-002-9224-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Mouelhi Guizani, S., Bouzaouach, I., Tenenbaum, G., Ben Kheder, A., Feki, Y., and Bouaziz, M. (2006). Simple and choice reaction times under varying levels of physical load in high skilled fencers. J. Sports Med. Phys. Fitness 46, 344–351.

Google Scholar

Mulert, C., Gallinat, J., Dorn, H., Herrmann, W. M., and Winterer, G. (2003). The relationship between reaction time, error rate and anterior cingulate cortex activity. Int. J. Psychophysiol. 47, 175–183. doi: 10.1016/S0167-8760(02)00125-3

CrossRef Full Text | Google Scholar

Muley, D. P. A., Wadikar, D. S. S., and Muley, D. P. P. (2016). Effect of exam Stress on reaction time in medical students (No. 4). Indian J. Basic Appl. Med. Res. 5, 733–739.

Google Scholar

Myles, W. S. (1985). Sleep deprivation, physical fatigue, and the perception of exercise intensity. Med. Sci. Sports Exerc. 17, 580–584. doi: 10.1249/00005768-198510000-00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Mysliwiec, V., McGraw, L., Pierce, R., Smith, P., Trapp, B., and Roth, B. J. (2013). Sleep disorders and associated medical comorbidities in active duty military personnel. Sleep 36, 167–174. doi: 10.5665/sleep.2364

PubMed Abstract | CrossRef Full Text | Google Scholar

Nafus, D. (2014). Stuck data, dead data, and disloyal data: the stops and starts in making numbers into social practices. Distinktion J. Soc. Theory 15, 208–222. doi: 10.1080/1600910X.2014.920266

CrossRef Full Text | Google Scholar

Nag, A., Mukhopadhyay, S. C., and Kosel, J. (2017). Wearable flexible sensors: a review. IEEE Sens. J. 17, 3949–3960. doi: 10.1109/JSEN.2017.2705700

CrossRef Full Text | Google Scholar

Naito, E., Kinomura, S., Geyer, S., Kawashima, R., Roland, P. E., and Zilles, K. (2000). Fast reaction to different sensory modalities activates common fields in the motor areas, but the anterior cingulate cortex is involved in the speed of reaction. J. Neurophysiol. 83, 1701–1709. doi: 10.1152/jn.2000.83.3.1701

PubMed Abstract | CrossRef Full Text | Google Scholar

National Research Council (2015). Measuring Human Capabilities: An Agenda for Basic Research on the Assessment of Individual and Group Performance Potential for Military Accession. Washington, DC: National Academies Press.

Google Scholar

Nilsson, J. P., Söderström, M., Karlsson, A. U., Lekander, M., Akerstedt, T., Lindroth, N. E., et al. (2005). Less effective executive functioning after one night’s sleep deprivation. J. Sleep Res. 14, 1–6. doi: 10.1111/j.1365-2869.2005.00442.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Nollet, M., Wisden, W., and Franks, N. P. (2020). Sleep deprivation and stress: a reciprocal relationship. Interface Focus 10:20190092. doi: 10.1098/rsfs.2019.0092

PubMed Abstract | CrossRef Full Text | Google Scholar

Okano, K., Kaczmarzyk, J. R., Dave, N., Gabrieli, J. D. E., and Grossman, J. C. (2019). Sleep quality, duration, and consistency are associated with better academic performance in college students. NPJ Sci. Learn. 4, 1–5. doi: 10.1038/s41539-019-0055-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Olver, J. S., Pinney, M., Maruff, P., and Norman, T. R. (2015). Impairments of spatial working memory and attention following acute psychosocial stress. Stress Health 31, 115–123. doi: 10.1002/smi.2533

PubMed Abstract | CrossRef Full Text | Google Scholar

Ozaki, H., Loenneke, J. P., Thiebaud, R. S., and Abe, T. (2013). Resistance training induced increase in VO 2 max in young and older subjects. Eur. Rev. Aging Phys. Activity 10, 107–116. doi: 10.1007/s11556-013-0120-1

CrossRef Full Text | Google Scholar

Padgett, D. A., and Glaser, R. (2003). How stress influences the immune response. Trends Immunol. 24, 444–448. doi: 10.1016/S1471-4906(03)00173-X

CrossRef Full Text | Google Scholar

Pahwa, A. R. R., Miller, D. J., Caplan, J. B., and Collins, D. F. (2020). Performance on an associative memory test decreases 8 hr after cardiovascular exercise. J. Sport Exerc. Psychol. 42, 219–226. doi: 10.1123/jsep.2019-0224

PubMed Abstract | CrossRef Full Text | Google Scholar

Palmer, C. J., Bigelow, C., and Emmerik, R. E. A. V. (2013). Defining soldier equipment trade space: load effects on combat marksmanship and perception–action coupling. Ergonomics 56, 1708–1721. doi: 10.1080/00140139.2013.832805

PubMed Abstract | CrossRef Full Text | Google Scholar

Parsons, I. T., Stacey, M. J., and Woods, D. R. (2019). Heat adaptation in military personnel: mitigating risk, maximizing performance. Front. Physiol. 10:1485. doi: 10.3389/fphys.2019.01485

PubMed Abstract | CrossRef Full Text | Google Scholar

Pavlov, Y. G., and Kotchoubey, B. (2021). Temporally distinct oscillatory codes of retention and manipulation of verbal working memory. BioRxiv [preprint] doi: 10.1101/2021.03.13.435253

CrossRef Full Text | Google Scholar

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., et al. (2011). Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830.

Google Scholar

Pick, H. L., and Palmer, C. F. (1986). “Perception and representation in the guidance of spatially coordinated behavior,” in Motor Development in Children: Aspects of Coordination and Control, eds M. G. Wade and H. T. A. Whiting (Leiden: Martinus Nijhoff Publishers), 135–145.

Google Scholar

Posada-Quintero, H. F., Reljin, N., Mills, C., Mills, I., Florian, J. P., VanHeest, J. L., et al. (2018). Time-varying analysis of electrodermal activity during exercise. PLoS One 13:e0198328. doi: 10.1371/journal.pone.0198328

PubMed Abstract | CrossRef Full Text | Google Scholar

Pyke, W., Ifram, F., Coventry, L., Sung, Y., Champion, I., and Javadi, A.-H. (2019). The effects of different protocols of physical exercise and rest on long-term memory. Neurobiol. Learn. Mem. 167:107128. doi: 10.31219/osf.io/ckvdf

CrossRef Full Text | Google Scholar

Rabbitt, P. (2004). Methodology of Frontal and Executive Function. Hove: Psychology Press.

Google Scholar

Robinson, C. S. H., Bryant, N. B., Maxwell, J. W., Jones, A. P., Robert, B., Lamphere, M., et al. (2018). The benefits of closed-loop transcranial alternating current stimulation on subjective sleep quality. Brain Sci. 8:204. doi: 10.3390/brainsci8120204

PubMed Abstract | CrossRef Full Text | Google Scholar

Robles, T. F., Shetty, V., Zigler, C. M., Glover, D. A., Elashoff, D., Murphy, D., et al. (2011). The feasibility of ambulatory biosensor measurement of salivary alpha amylase: relationships with self-reported and naturalistic psychological stress. Biol. Psychol. 86, 50–56. doi: 10.1016/j.biopsycho.2010.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Rodas, J. A., and Greene, C. (2020). Working memory training does not improve executive functioning or fluid intelligence. PsyArXiv [preprint] doi: 10.31234/osf.io/8nh35

CrossRef Full Text | Google Scholar

Rodeback, R. E., Hedges-Muncy, A., Hunt, I. J., Carbine, K. A., Steffen, P. R., and Larson, M. J. (2020). The association between experimentally induced stress, performance monitoring, and response inhibition: an event-related potential (ERP) analysis. Front. Hum. Neurosci. 14:189. doi: 10.3389/fnhum.2020.00189

PubMed Abstract | CrossRef Full Text | Google Scholar

Roozendaal, B., McEwen, B. S., and Chattarji, S. (2009). Stress, memory and the amygdala. Nat. Rev. Neurosci. 10, 423–433. doi: 10.1038/nrn2651

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosenbaum, D., Mama, Y., and Algom, D. (2017). Stand by your stroop: standing up enhances selective attention and cognitive control. Psychol. Sci. 28, 1864–1867. doi: 10.1177/0956797617721270

PubMed Abstract | CrossRef Full Text | Google Scholar

Sabes, P. N., and Jordan, M. I. (1997). Obstacle avoidance and a perturbation sensitivity model for motor planning. J. Neurosci. 17, 7119–7128. doi: 10.1523/JNEUROSCI.17-18-07119.1997

PubMed Abstract | CrossRef Full Text | Google Scholar

Sanabria, D., Luque-Casado, A., Perales, J. C., Ballester, R., Ciria, L. F., Huertas, F., et al. (2019). The relationship between vigilance capacity and physical exercise: a mixed-effects multistudy analysis. PeerJ 7:e7118. doi: 10.7717/peerj.7118

PubMed Abstract | CrossRef Full Text | Google Scholar

Sandi, C. (2013). Stress and cognition. WIREs Cogn. Sci. 4, 245–261. doi: 10.1002/wcs.1222

PubMed Abstract | CrossRef Full Text | Google Scholar

Sandra, D. A., and Otto, A. R. (2018). Cognitive capacity limitations and Need for Cognition differentially predict reward-induced cognitive effort expenditure. Cognition 172, 101–106. doi: 10.1016/j.cognition.2017.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Sapolsky, R. M., Romero, L. M., and Munck, A. U. (2000). How do glucocorticoids influence stress responses? Integrating permissive, suppressive, stimulatory, and preparative actions. Endoc. Rev. 21, 55–89. doi: 10.1210/edrv.21.1.0389

PubMed Abstract | CrossRef Full Text | Google Scholar

Scheinost, D., Noble, S., Horien, C., Greene, A. S., Lake, E. M. R., Salehi, M., et al. (2019). Ten simple rules for predictive modeling of individual differences in neuroimaging. NeuroImage 193, 35–45. doi: 10.1016/j.neuroimage.2019.02.057

PubMed Abstract | CrossRef Full Text | Google Scholar

Scheit, L. (2021). Optimizing the introduction of wearable sensors into the german armed forces for military medical applications. Mil. Med. 186, 962–968. doi: 10.1093/milmed/usab015

PubMed Abstract | CrossRef Full Text | Google Scholar

Schumacher, N., Schmidt, M., Wellmann, K., and Braumann, K.-M. (2018). General perceptual-cognitive abilities: age and position in soccer. PLoS One 13:e0202627. doi: 10.1371/journal.pone.0202627

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwabe, L., Joëls, M., Roozendaal, B., Wolf, O. T., and Oitzl, M. S. (2012). Stress effects on memory: an update and integration. Neurosci. Biobehav. Rev. 36, 1740–1749. doi: 10.1016/j.neubiorev.2011.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Scott, J. P. R., McNaughton, L. R., and Polman, R. C. J. (2006). Effects of sleep deprivation and exercise on cognitive, motor performance and mood. Physiol. Behav. 87, 396–408. doi: 10.1016/j.physbeh.2005.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Scribbans, T. D., Vecsey, S., Hankinson, P. B., Foster, W. S., and Gurd, B. J. (2016). The effect of training intensity on VO2max in young healthy adults: a meta-regression and meta-analysis. Int. J. Exerc. Sci. 9, 230–247.

Google Scholar

Scribner, D. R. (2016). Predictors of shoot–don’t shoot decision-making performance: an examination of cognitive and emotional factors. J. Cogn. Eng. Decis. Making 10, 3–13. doi: 10.1177/1555343415608974

CrossRef Full Text | Google Scholar

Seeley, W. W., Menon, V., Schatzberg, A. F., Keller, J., Glover, G. H., Kenna, H., et al. (2007). Dissociable intrinsic connectivity networks for salience processing and executive control. J. Neurosci. 27, 2349–2356. doi: 10.1523/JNEUROSCI.5587-06.2007

PubMed Abstract | CrossRef Full Text | Google Scholar

Seo, K., Kim, K., Park, Y. J., Cho, J.-K., Lee, J., Choi, B., et al. (2018). “Adaptive oscillator-based control for active lower-limb exoskeleton and its metabolic impact,” in Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), (Piscataway, NY: IEEE), 6752–6758. doi: 10.1109/ICRA.2018.8460841

CrossRef Full Text | Google Scholar

Shan, Y., Li, S., and Chen, T. (2020). Respiratory signal and human stress: non-contact detection of stress with a low-cost depth sensing camera. Int. J. Mach. Learn. Cybern. 11, 1825–1837. doi: 10.1007/s13042-020-01074-x

CrossRef Full Text | Google Scholar

Sheffield, B., Ziriax, J., Keller, M. D., Barns, W., and Brungart, D. (2017). The impact of reduced speech intelligibility on reaction time in a naval combat environment. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 61, 1570–1574. doi: 10.1177/1541931213601757

CrossRef Full Text | Google Scholar

Shepertycky, M., Burton, S., Dickson, A., Liu, Y.-F., and Li, Q. (2021). Removing energy with an exoskeleton reduces the metabolic cost of walking. Science 372, 957–960. doi: 10.1126/science.aba9947

PubMed Abstract | CrossRef Full Text | Google Scholar

Shields, G. S., Bonner, J., and Moons, W. (2015). Does cortisol influence core executive functions? A meta-analysis of acute cortisol administration effects on working memory, inhibition, and set-shifting. Psychoneuroendocrinology 58, 91–103. doi: 10.1016/j.psyneuen.2015.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

Shields, G. S., Sazma, M. A., and Yonelinas, A. P. (2016). The effects of acute stress on core executive functions: a meta-analysis and comparison with cortisol. Neurosci. Biobehav. Rev. 68, 651–668. doi: 10.1016/j.neubiorev.2016.06.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Sibley, B. A., and Etnier, J. L. (2003). The relationship between physical activity and cognition in children: a meta-analysis. Pediatr. Exerc. Sci. 15, 243–256. doi: 10.1123/pes.15.3.243

CrossRef Full Text | Google Scholar

Singh, A., Kaushik, A., Kumar, R., Nair, M., and Bhansali, S. (2014). Electrochemical sensing of cortisol: a recent update. Appl. Biochem. Biotechnol. 174, 1115–1126. doi: 10.1007/s12010-014-0894-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Smith, T. J., Henning, R. A., Wade, M. G., and Fisher, T. (2014). Variability in Human Performance. Boca Raton, FL: CRC Press.

Google Scholar

Solis, M. Y., Cooper, S., Hobson, R. M., Artioli, G. G., Otaduy, M. C., Roschel, H., et al. (2015). Effects of beta-alanine supplementation on brain homocarnosine/carnosine signal and cognitive function: an exploratory study. PLoS One 10:e0123857. doi: 10.1371/journal.pone.0123857

PubMed Abstract | CrossRef Full Text | Google Scholar

Spiegel, K., Leproult, R., and Van Cauter, E. (1999). Impact of sleep debt on metabolic and endocrine function. Lancet (London, England) 354, 1435–1439. doi: 10.1016/S0140-6736(99)01376-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Spiegel, K., Leproult, R., and Van Cauter, E. (2016). “Metabolic and endocrine changes,” in Sleep Deprivation: Basic Science, Physiology and Behavior, ed. C. A. Kushida (Boca Raton, FL: CRC Press), 293–318.

Google Scholar

Stanley, E. A., Schaldach, J. M., Kiyonaga, A., and Jha, A. P. (2011). Mindfulness-based mind fitness training: a case study of a high-stress predeployment military cohort. Cogn. Behav. Pract. 18, 566–576. doi: 10.1016/j.cbpra.2010.08.002

CrossRef Full Text | Google Scholar

Starcke, K., and Brand, M. (2016). Effects of stress on decisions under uncertainty: a meta-analysis. Psychol. Bull. 142, 909–933. doi: 10.1037/bul0000060

PubMed Abstract | CrossRef Full Text | Google Scholar

Swets, J. A., and Bjork, R. A. (1990). Enhancing human performance: an evaluation of “new age” techniques considered by the U.S. Army. Psychol. Sci. 1, 85–96. doi: 10.1111/j.1467-9280.1990.tb00074.x

CrossRef Full Text | Google Scholar

Taheri, M., and Irandoust, K. (2020). Morning exercise improves cognitive performance decrements induced by partial sleep deprivation in elite athletes. Biol. Rhythm Res. 51, 644–653. doi: 10.1080/09291016.2019.1576279

CrossRef Full Text | Google Scholar

Tanaka, Y., Fujimura, N., Tsuji, T., Maruishi, M., Muranaka, H., and Kasai, T. (2009). Functional interactions between the cerebellum and the premotor cortex for error correction during the slow rate force production task: an fMRI study. Exp. Brain Res. 193, 143–150. doi: 10.1007/s00221-008-1682-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Temesi, J., Arnal, P. J., Davranche, K., Bonnefoy, R., Levy, P., Verges, S., et al. (2013). Does central fatigue explain reduced cycling after complete sleep deprivation? Med. Sci. Sports Exerc. 45, 2243–2253. doi: 10.1249/MSS.0b013e31829ce379

PubMed Abstract | CrossRef Full Text | Google Scholar

Terrin, N., Schmid, C. H., Griffith, J. L., D’Agostino, R. B., and Selker, H. P. (2003). External validity of predictive models: a comparison of logistic regression, classification trees, and neural networks. J. Clin. Epidemiol. 56, 721–729. doi: 10.1016/S0895-4356(03)00120-3

CrossRef Full Text | Google Scholar

Terse-Thakoor, T., Punjiya, M., Matharu, Z., Lyu, B., Ahmad, M., Giles, G. E., et al. (2020). Thread-based multiplexed sensor patch for real-time sweat monitoring. NPJ Flex. Electron. 4, 1–10. doi: 10.1038/s41528-020-00081-w

CrossRef Full Text | Google Scholar

Tharion, W. J., Montain, S. J., O’Brien, C., Shippee, R. L., and Hoban, J. L. (1997). Effects of military exercise tasks and a carbohydrate-electrolyte drink on rifle shooting performance in two shooting positions. Int. J. Ind. Ergon. 19, 31–39. doi: 10.1016/0169-8141(95)00087-9

CrossRef Full Text | Google Scholar

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288. doi: 10.1111/j.2517-6161.1996.tb02080.x

CrossRef Full Text | Google Scholar

Timme, S., and Brand, R. (2020). Affect and exertion during incremental physical exercise: examining changes using automated facial action analysis and experiential self-report. PLoS One 15:e0228739. doi: 10.1371/journal.pone.0228739

PubMed Abstract | CrossRef Full Text | Google Scholar

Trommershäuser, J., Maloney, L. T., and Landy, M. S. (2003). Statistical decision theory and the selection of rapid, goal-directed movements. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1419–1433. doi: 10.1364/JOSAA.20.001419

PubMed Abstract | CrossRef Full Text | Google Scholar

Truszczynski, O., Lewkowicz, R., Wojtkowiak, M., and Biernacki, M. P. (2014). Reaction time in pilots during intervals of high sustained G. Aviat. Space Environ. Med. 85, 1114–1120. doi: 10.3357/ASEM.4009.2014

PubMed Abstract | CrossRef Full Text | Google Scholar

Tseng, P., Napier, B., Garbarini, L., Kaplan, D. L., and Omenetto, F. G. (2018). Functional, RF-trilayer sensors for tooth-mounted, wireless monitoring of the oral cavity and food consumption. Adv. Mater. 30:1703257. doi: 10.1002/adma.201703257

PubMed Abstract | CrossRef Full Text | Google Scholar

Tsukahara, J. S., Harrison, T. L., Draheim, C., Martin, J. D., and Engle, R. W. (2020). Attention control: the missing link between sensory discrimination and intelligence. Attent. Percept. Psychophys. 82, 3445–3478. doi: 10.3758/s13414-020-02044-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Upasham, S., Tanak, A., Jagannath, B., and Prasad, S. (2018). Development of ultra-low volume, multi-bio fluid, cortisol sensing platform. Sci. Rep. 8:16745. doi: 10.1038/s41598-018-35199-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Van Helder, T., and Radomski, M. W. (1989). Sleep deprivation and the effect on exercise performance. Sports Med. 7, 235–247. doi: 10.2165/00007256-198907040-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

Vergouwe, Y., Moons, K. G. M., and Steyerberg, E. W. (2010). External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am. J. Epidemiol. 172, 971–980. doi: 10.1093/aje/kwq223

PubMed Abstract | CrossRef Full Text | Google Scholar

Verhagen, L., Dijkerman, H. C., Grol, M. J., and Toni, I. (2008). Perceptuo-Motor interactions during prehension movements. J. Neurosci. 28, 4726–4735. doi: 10.1523/JNEUROSCI.0057-08.2008

PubMed Abstract | CrossRef Full Text | Google Scholar

Vine, C., Coakley, S., Myers, S., Blacker, S., and Runswick, O. (2020). The reliability of a military specific auditory n-back task and shoot/don’t-shoot task. PsyArXiv [preprint] doi: 10.31234/osf.io/89vb5

CrossRef Full Text | Google Scholar

von Helversen, B., and Rieskamp, J. (2020). Stress-related changes in financial risk taking: considering joint effects of cortisol and affect. Psychophysiology 57:e13560. doi: 10.1111/psyp.13560

PubMed Abstract | CrossRef Full Text | Google Scholar

Vrijkotte, S., Roelands, B., Meeusen, R., and Pattyn, N. (2016). Sustained military operations and cognitive performance. Aerospace Med. Hum. Perform. 87, 718–727. doi: 10.3357/AMHP.4468.2016

PubMed Abstract | CrossRef Full Text | Google Scholar

Wei, N., Zhou, T., Zhang, Z., Zhuo, Y., and Chen, L. (2019). Visual working memory representation as a topological defined perceptual object. J. Vis. 19:12. doi: 10.1167/19.7.12

CrossRef Full Text | Google Scholar

Wei, W., Wang, W., Qu, Z., Gu, J., Lin, X., and Yue, C. (2020). The effects of a passive exoskeleton on muscle activity and metabolic cost of energy. Adv. Robot. 34, 19–27. doi: 10.1080/01691864.2019.1707708

CrossRef Full Text | Google Scholar

Welford, A. T. (1980). “Choice reaction time: basic concepts,” in Reaction Times, ed. A. T. Welford (London: Academic Press), 73–128.

Google Scholar

Willoughby, M., Hong, Y., Hudson, K., and Wylie, A. (2020). Between- and within-person contributions of simple reaction time to executive function skills in early childhood. J. Exp. Child Psychol. 192:104779. doi: 10.1016/j.jecp.2019.104779

PubMed Abstract | CrossRef Full Text | Google Scholar

Windmiller, J. R., and Wang, J. (2013). Wearable electrochemical sensors and biosensors: a review. Electroanalysis 25, 29–46. doi: 10.1002/elan.201200349

CrossRef Full Text | Google Scholar

Wohl, J. G. (1981). Force management decision requirements for air force tactical command and control. IEEE Trans. Syst. Man Cybern. 11, 618–639. doi: 10.1109/TSMC.1981.4308760

CrossRef Full Text | Google Scholar

Woodward, R. B., Stokes, M. J., Shefelbine, S. J., and Vaidyanathan, R. (2019). Segmenting mechanomyography measures of muscle activity phases using inertial data. Sci. Rep. 9:5569. doi: 10.1038/s41598-019-41860-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Yates, R. D., and Goodman, D. J. (2014). Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers. New York, NY: John Wiley & Sons.

Google Scholar

Yoo, E.-H., and Lee, S.-Y. (2010). Glucose biosensors: an overview of use in clinical practice. Sensors 10, 4558–4576. doi: 10.3390/s100504558

PubMed Abstract | CrossRef Full Text | Google Scholar

Zakrzewski, M., Raittinen, H., and Vanhala, J. (2012). Comparison of center estimation algorithms for heart and respiration monitoring with microwave doppler radar. IEEE Sens. J. 12, 627–634. doi: 10.1109/JSEN.2011.2119299

CrossRef Full Text | Google Scholar