Negative valuation of ambiguous feedback may predict near-term risk for suicide attempt in Veterans at high risk for suicide

Background: Learning from feedback – adapting behavior based on reinforcing and punishing outcomes – has been implicated in numerous psychiatric disorders, including substance misuse, post-traumatic stress disorder, and depression; an emerging literature suggests it may also play a role in suicidality. This study examined whether a feedback-based learning task with rewarding, punishing and ambiguous outcomes, followed by computational modeling, could improve near-term prospective prediction of suicide attempt in a high-risk sample.

Method: Veterans (N=60) at high-risk for suicide were tested on a task of reward- and punishment-based learning, at multiple sessions across a one-year period. Each session was coded according to whether the participant had (1) an actual suicide attempt (ASA); (2) another suicide-related event (OtherSE) such as suicidal behavior or suicidal ideation-related hospital admission (but not an ASA); or (3) neither (noSE) in the next 90 days. Computational modeling was used to estimate latent cognitive variables including learning rates from positive and negative outcomes, and the subjective value of ambiguous feedback.

Results: Optimal responding on the reward-based trials was positively associated with upcoming ASA, and remained predictive even after controlling for other standard clinical variables such as current suicidal ideation severity and prior suicide attempts. Computational modeling revealed that patients with upcoming ASA tended to view ambiguous outcomes as similar to weak punishment, while OtherSE and noSE both tended to view the ambiguous outcome as similar to weak reward. Differences in the reinforcement value of the neutral outcome remained predictive for ASA even after controlling for current suicidal ideation and prior suicide attempts.

Conclusion: A reinforcement learning task with ambiguous neutral outcomes may provide a useful tool to help predict near-term risk of ASA in at-risk patients. While most individuals interpret ambiguous feedback as mildly reinforcing (a “glass half full” interpretation), those with upcoming ASA tend to view it as mildly punishing (a “glass half empty” interpretation). While the current results are based on a very small sample with relatively few ASA events, and require replication in a larger sample, they provide support for the role of negative biases in feedback-based learning in the cognitive profile of suicide risk.

1 Introduction

Death by suicide is a public health epidemic across demographic and geographic groups, with increasing rates among younger adults as well as Veterans and active military members (1), despite considerable investments in research, policy, and clinical programs (2). Although there has been intensive study of epidemiological risk factors for suicide (3), an emerging priority is the identification not only of which individuals are at risk, but also when that risk is highest (4), so that clinical and supportive interventions can be applied in a timely and effective manner. For this reason, recent studies have begun to consider near-term factors that indicate upcoming risk within a more acute time window of days to weeks (5), as well as examining cognitive changes that underlie an individual’s transition from suicidal ideation to actual suicide attempt (6).

One important aspect of cognition is feedback-based learning. Individuals adjust their behavior by learning from reinforcing and punishing outcomes, in order to adapt to daily life circumstances (7). Altered feedback-based learning has been implicated in a number of psychiatric disorders including addiction (8), post-traumatic stress disorder (9), and anxiety and depression (10), and may also be a pathway contributing to suicidal behavior. In one prior study, older adult participants learned responses through trial-and-error, with variable reward rates and contingencies; results suggested that that depressed individuals with history of suicide attempt showed a tendency to discount prior feedback in favor of more recent feedback (11). Such learning biases could be linked to suicidality by leading individuals to discount historical experiences, and instead focus on more recent negative events in guiding their current behavior. Findings such as these are consistent with the role of positive valence systems (including reward responsiveness, reward learning, and reward valuation) in suicide-related psychopathology (12). Feedback-based learning tasks therefore represent a potential inroad to understanding the cognitive changes that underlie risk for suicide.

Feedback-based learning tasks also lend themselves to computational modeling, which can provide an explanatory link from behavior to the psychobiological mechanisms underlying those observable behaviors (13, 14). Reinforcement-learning (RL) models learn through trial-and-error to adapt their behavior in order to maximize rewards and avoid punishments. Importantly, the key teaching signal in RL models – prediction error – has been associated with mesolimbic dopamine systems (15), linking the latent cognitive variables extracted by RL models with the brain substrates of learning (16). Blunting of prediction error has been observed in depressed patients with a history of suicide attempt, and is correlated with blunted value signals in the ventromedial prefrontal cortex (17).

Applied to individual-level data, RL models can be used to discover a set of model parameters that together best reproduce each participant’s trial-by-trial learning behavior; the parameters correspond to latent cognitive variables such as learning rate to better-than-expected and worse-than-expected outcomes, tendency to exploit previously-successful strategies versus occasionally explore new ones, and the subjective value of reinforcement (18). These latent cognitive variables may be altered in neurological and psychiatric patient groups, thus identifying potential mechanisms that could be driving group differences in behavior (19). For example, one study reported that depressed individuals with a history of suicide attempt had increased learning rate on trials with worse-than-expected outcomes, but only after negative mood induction (20).

However, many existing feedback-based learning tasks conflate learning to obtain reward with learning to avoid punishment, despite the fact that these appear to be distinct systems that can be differentially affected in psychopathology. To dissociate reward-based and punishment-based learning, the probabilistic reward- and punishment-based learning task (RPLT) interleaves reward-based training, which provides positively-valenced feedback for correct responses and no feedback for incorrect responses, with punishment training, which provides negatively-valenced feedback for incorrect responses and no feedback for correct responses (21–24). This allows for separate consideration of reward-based learning and punishment-based learning within a single participant. Thus, for example, the RPLT has been used to show selective impairment on reward-based learning but relative sparing of punishment-based learning in unmedicated patients with depressive disorders (25, 26); unmedicated patients with Parkinson’s disease (PD) show a similar pattern, but dopaminergic medication reverses this pattern, leading to remediated reward-based learning but impaired punishment-based learning (24).

The RPLT also allows the use of RL modeling to examine the subjective reinforcement value of the no-feedback outcome (R₀), which is ambiguous as it could represent either a failure to obtain reward or else a successful avoidance of punishment. For example, one prior study with the RPLT found that Veterans with severe post-traumatic stress disorder (PTSD) symptoms showed better reward-based learning (but not punishment-based learning), compared to peers with few or no PTSD symptoms; the PTSD group also tended to value R₀ significantly less positively (21), exaggerating the perceived difference between reward and non-reward in the task. Thus, RL modeling provided an explanation for the seemingly paradoxical facilitation of reward-based learning in the PTSD group, but also suggested that negatively-biased subjective valuation of ambiguous or neutral events could contribute to PTSD symptoms.

The current study was an initial evaluation of use of the RPLT to prospectively predict suicide attempt within the subsequent 90 days, in a small group of Veterans at risk for suicide. This study builds on the existing literature in two important ways. First, much of the existing literature relating suicidality and learning has employed retrospective research designs, comparing participants with vs. without a history of prior suicide attempt; in some cases, attempts may have occurred years or even decades previously. In addition to focusing on prospective prediction of suicide attempt, the current study builds on suicide risk factor research more broadly by evaluating a much-needed shorter window of suicide risk: 90 days. While this can be challenging given the low base rate of suicide behavior, we pursued this goal by recruiting a sample of Veterans at high-risk for suicide within one year post a suicide-related hospital admission, and by conducting multiple assessments over a one-year follow-up period. Second, the current study builds on previous research evaluating the role of feedback-based learning and suicide risk (11, 17), by utilizing a task that allows for separate evaluation of learning from reward vs. punishment, and of the subjective valuation of ambiguous neutral feedback.

Our primary hypothesis was that performance on the RPLT could be used to predict near-term (90-day) suicide attempt, above and beyond standard variables used in suicide risk assessment (i.e., suicidal ideation severity and number of prior suicide attempts). Our secondary hypothesis was that latent cognitive variables, extracted from RL modeling of the behavioral data, such as learning rate from rewarding or punishing outcomes and/or the subjective valuation of neutral feedback, would also predict upcoming suicide attempt, indicating specific cognitive processes altered in at-risk individuals entering a period of heightened risk for suicide attempt.

2 Materials and methods

2.1 Participants

Participants were N = 60 Veterans receiving care through the VA New Jersey Health Care System (VANJHCS) with a history of acute psychiatry admission due to suicide attempt (SA) or severe suicidal ideation (SI) during the prior year. Other inclusion criteria were at least one of the following: (a) actual, interrupted, or aborted suicide attempt during the prior year, as measured by the Columbia Suicide Severity Rating Scale (C-SSRS) (27); or (b) one or more instances of preparatory behavior in the prior year, defined as actions directed toward a suicide attempt that go beyond ideation, such as assembling a specific method (e.g. buying pills, purchasing a gun) or preparing for one’s death by suicide (e.g. giving things away, writing a farewell note); or (c) clinically-significant SI within the prior week, defined as a score of 4+ on the Beck Scale for Suicidal Ideation (SSI) (28). Participants were screened for cognitive capacity using the Montreal Cognitive Assessment (MoCA) (29) and for history of psychiatric disorders using the Mini International Neuropsychiatric Interview (MINI) (30); however, current or past medical or psychiatric diagnoses did not automatically trigger study exclusion if participants were otherwise capable of participating. Based on these inclusion criteria, a transdiagnostic group of participants were recruited, with substance use, head injury, PTSD, and mood disorders being the most common psychiatric difficulties. Full recruitment and screening details for the study are provided in the Appendix.

Participants who met enrollment criteria provided written informed consent; at this point, information from screening (including C-SSRS, SSI, MoCA, MINI) became part of the study data. The study was approved and monitored by the VANJHCS Institutional Review Board (protocol #1577294) and conformed to the Declaration of Helsinki and U.S. Federal policy for the protection of human subjects.

2.2 Procedures

At baseline (Session 1), participants completed a clinical interview, several self-report questionnaires, and the reward- and punishment-learning task (RPLT) described further below. The clinical interview included updated C-SSRS and SSI (if screening had not occurred on the same day), brief medical history including history of traumatic brain injury (TBI), using the Brief Traumatic Brain Injury Screen (BTBIS) (31) modified to include injuries sustained outside of as well as during military deployment, and combat exposure, using the Combat Exposure Scale (CES) (32). Self-report questionnaires included the Beck Depression Inventory (BDI-II) (33) to assess severity of depression symptoms in previous two weeks; and the Beck Hopelessness Scale (BHS) (34) to assess hopelessness and negative attitudes about the future during the past week. Full details of data collection methods are provided in the Appendix.

Participants also completed the reward- and punishment-learning task (RPLT), described below; most also completed additional questionnaires and neurocognitive tasks not reported here. In most cases, neurocognitive testing occurred either immediately after collection of clinical and self-report data (80.5%) or within 24h (11.9%).

Following baseline testing, participants were followed for one year, with follow-up testing (Sessions 2-5) occurring at approximately 3-month intervals. Follow-up sessions included updated clinical interviews (including C-SSRS and SSI), self-report questionnaires, and neurocognitive tasks. In cases where individuals could not travel to the testing site for follow-up testing (e.g., during the COVID-19 pandemic, Spring 2020-Spring 2021), efforts were made to collect interview and questionnaire data by telephone and/or mail, especially updated C-SSRS and SSI, allowing capture of suicide-related events during the follow-up period.

2.3 Reward and punishment-learning task

The RPLT was adapted from previously described methods (21, 22), programmed using PsychoPy (35) and presented on a Dell laptop or similar computer, with the subject seated in a quiet room at a comfortable viewing distance from the computer. The RPLT task software is available at: https://osf.io/p328a/.

In brief, the RPLT is a probabilistic categorization task, in which participants learn through trial-and-error to classify each of four stimuli into categories A and B. On each trial, one of four stimuli (S1, S2, S3, S4) appears, and the subject is asked to guess whether the stimulus belongs to category A or B, by pressing one of two keyboard keys. The visual images for stimuli S1-S4 are from the Novel Object and Unusual Name (NOUN) Database, 2^nd edition (36), downloaded from http://sussex.ac.uk/wordlab/noun (accessed March 2019); a different set of four images is used at each testing session.

Stimuli S1 and S2 are reward-based stimuli, meaning that a correct categorization is rewarded by on-screen feedback (smiley face) and point gain (25 points), while an incorrect categorization receives no feedback (Figure 1-top). Stimuli S3 and S4 are punishment-based stimuli, meaning that an incorrect categorization is punished by on-screen feedback (frowning face) and point loss (25 points), while correct responses trigger no feedback (Figure 1-bottom). Thus, the no-feedback outcome is ambiguous, since on some trials it signals missed opportunity to obtain reward, and on others it signals successful avoidance of punishment. The task thus allows within-subject comparison of learning from reward vs. punishment. Participants are not explicitly informed whether each trial is a reward-based or feedback-based trial, and therefore whether the ambiguous (no-feedback) outcome represents a missed reward or a missed loss, although participants can learn across trials which stimuli are associated with potential reward vs. potential loss, and therefore infer the meaning of the no-feedback outcome when it occurs.

Figure 1

Figure 1. Screenshots from RPLT including (top left) a reward-based trial where the participant responds correctly and wins points (top right), and (bottom left) a punishment-based trial where the participant responds incorrectly and loses points (bottom left). A running point tally appears at lower right of screen. Different stimuli are used at each testing session.

Category mappings are probabilistic. Stimuli S1 and S3 belong to category A on 80% of trials and to category B on 20% of trials; stimuli S2 and S4 belong to category B on 80% of trials and category A on 20% of trials. The optimal response for each stimulus is therefore the most frequently correct category, regardless of which response is actually correct on a given trial.

Before the first trial, there is a short practice phase (using two stimuli that do not appear during the regular task trials); participants are guided through responding to a reward-based stimulus (once to obtain reward, and once with no feedback) and a punishment-based stimulus (once receiving punishment, and once with no feedback). Participants are instructed to try to maximize point tally and are reminded that the same stimulus does not necessarily belong to the same category every time it appears. They may use either (or both) hands to enter responses.

The task itself comprises 80 trials, including two blocks of 40 trials; each block contains 10 trials with each stimulus, 8 with the more frequent category and 2 with the less frequent category. Trial order within a block is pseudorandom but fixed across subjects, with the constraint that each stimulus is always paired with its more frequently-correct category on the first trial where it appears.

There is no time limit for responding. Responses occurring <200 msec after stimulus onset are discarded as anticipatory responses; the program records the first response after 200 msec, which ends the trial. There is an inter-trial interval of 250 msec, during which the point tally remains visible, before the next stimulus appears. The tally is initialized to 500 points at the start of practice, to ensure the tally does not drop below 0 during the course of the experiment, which could induce subject frustration.

For each trial, the computer records the stimulus, the participant response and whether that was the optimal response (regardless of actual outcome), along with reaction time and any on-screen feedback. Summary behavioral variables analyzed for each subject were percent optimal responses to reward-based and punishment-based stimuli.

2.4 Computational modeling

The behavioral data (trial-by-trial stimuli, responses, and feedback) were also used as input to a reinforcement learning (RL) model (7, 18). Following previously-described methods (22, 37), we used a Q-learning model (38, 39), which learns an expected value Q for each possible response to the current stimulus; on each trial, the model examines the Q-values to choose a response, and then updates Q-values based on prediction error (PE), defined as the difference between the expected outcome and the actual reinforcement received. Model performance is defined by parameters including the learning rate on trials with better-than-expected outcomes (LR+), learning rate for trials with worse-than-expected outcomes (LR-), the tendency to exploit previously-successful responses vs. occasionally explore new ones (β), and the subjective value of ambiguous reinforcement (R₀). Individual-level data are simulated by optimizing parameter values such that the model’s trial-by-trial behavior most closely mimics the participant’s trial-by-trial choices; the resulting configuration of parameters represent estimates of latent cognitive variables for that participant. Full details of model building and testing (including predictive check and parameter recovery studies) appear in the Appendix.

2.5 Prospective outcomes

Following prior methods (40, 41), for each RPLT testing session, we categorized near-term clinical outcomes into one of three mutually exclusive categories, based on previously-published case classification criteria (42): (1) “ASA” if the participant had 1+ actual suicide attempt (ASA), defined as deliberate self-harm with at least some intent of ending one’s life, within the 90-day period following RPLT testing; (2) “OtherSE” if there was no ASA within this window but at least one suicide-related event (SE) which could include interrupted/aborted suicide attempt, defined as acts of deliberate self-harm, with at least some suicidal intent, that are stopped prior to reaching a risk threshold; preparatory behavior such as assembling a method or preparing for one’s death by suicide; or severe suicidal ideation resulting in inpatient or emergency room admission; or (3) “noSE” if there was neither ASA nor other SE within the 90-day follow-up window.

ASAs and other SEs were determined from clinician-administered C-SSRS at all available testing sessions, supplemented by medical chart review. Participants were censored if they died or withdrew from the study (or were withdrawn by the study team, e.g., due to criminal justice involvement) during the 1-year follow-up period; data collected and outcomes occurring prior to censoring remained part of the study record. Participants who were lost to contact during the follow-up period were not considered censored as long as outcomes could be tracked through Veterans Health Administration (VHA) electronic medical records.

2.6 Statistical analysis

Generalized estimating equations (GEEs) were used to test the effects of predictors on the multinomial dependent variable. GEEs are an extension of generalized linear models for analysis of repeated measures with non-normal response variables (43–45), and can model longitudinal designs while accounting for correlations between observations within individuals (46–48). Here, the outcome variable had three levels: ASA, OtherSE, or noSE (reference level). Given that the same patient could contribute multiple data points (1-5 observations per subject), we accounted for within-subject clustering. Results are presented by exponentiating the coefficients from the estimated marginal model to obtain odds ratios (OR), with 95% confidence interval (CI); threshold for significance was set at .05.

GEE models were estimated using the nomLORgee function for GEE with multinomial outcomes, from the multgee package (49) in R v.4.3.1 (50). First, a simple model examined whether outcomes could be predicted by the RPLT behavioral scores (percent optimal responding to reward- and punishment-based stimuli). Second, to test our hypothesis regarding predictive value of behavioral measures, we used a GEE to evaluate the incremental utility of RPLT behavioral scores in predicting the response variable, over standard suicide risk variables (number of lifetime ASAs, SSI at time of testing) (40, 41, 51, 52).

To explore the latent cognitive variables derived from computational modeling, the same methods were used as for the behavioral data, but including the RL parameter estimates for each subject as predictors.

As supplemental analyses, we also examined other pertinent covariates, including PTSD, which has been shown to associate with improved reward-based learning and decreased R₀ on this task (21); depression, which has been shown to associate with impaired reward-based learning on this task (25, 26); opioid use; history of traumatic brain injury (TBI); and prior week SE, given that some of our participants were tested while in-patients in the immediate aftermath of an SE, while others were tested after discharge (though within one year of SI-related hospitalization). These supplemental analyses are reported in the Appendix.

3 Results

3.1 Sample characteristics

The sample of 60 Veterans included 7 females (11.7%), and had mean age 45.3 years (SD 13.8, range 23 to 72) at baseline, with average CES score of 9.2 (SD 10.9, range 0 to 39).

A majority of participants (47 of 60) had a prior history of suicide attempt, with n=20 (33.3%) reporting one prior ASA and n=27 (45.0%) reporting multiple (two or more) prior ASAs. Almost all (n=57, 95.0%) endorsed lifetime history of suicidal ideation with some intent, based on C-SSRS ideation severity of 4+, assessed at screening and baseline. About a third (n=22, 36.7%) had lifetime history of non-suicidal self-injurious behavior (NSSI). Table 1 summarizes other demographic and clinical information for the 60 Veteran participants.

Table 1

Table 1. Demographic and clinical information for 60 Veteran participants at baseline (Session 1).

During the one-year follow-up period, 4 individuals had one or more ASAs, and 17 had other suicide-related events (OtherSE), including 1 with an interrupted/aborted suicide attempt, 9 with one or more instances of preparatory behavior (but no attempt) and 7 with one or more SI-related hospital admissions (without attempt or preparatory behavior). Five participants were censored, including 2 who died of natural causes (at 205 and 217 days) and 3 withdrawn (1 by request of subject at 77 days, and 2 by study team due to criminal justice involvement at 86 and 125 days). One of the two individuals who died had an ASA at 180 days (more than 90 days after their first and only RPLT testing session); none of the other censored individuals had any suicide-related events before censoring. Data from censored individuals were included in the current analysis as long as censoring occurred >90 days after an RPLT testing session.

Of the 21 participants who had ASAs or other SEs during the one-year follow-up, many had multiple events. In total, there were 7 ASAs, 1 interrupted/aborted attempt, 15 instances of preparatory behavior, and 22 SI-related hospitalizations (excluding hospitalizations that occurred secondary to suicide attempt or preparatory behavior, to avoid double-counting).

3.2 RPLT and suicidal outcomes

A total of 144 RPLT sessions were completed. Of these, 21 were excluded because they could not be associated with an outcome due to censoring (study end <90 days after testing, n=16; study withdrawal, n=3; death from natural causes, n=2). An additional 5 files from two participants were dropped due to apparent noncompliance (e.g., participant only pressed one response key during the entire RPLT task). In three additional cases, participants completed only the first 50-63 trials (before computer crash or other interruption), but the available trials were retained and scored.

Thus, 118 files from 58 unique participants were ultimately analyzed. There were no obvious differences in demographics, clinical profile, or outcome distribution among the dropped files compared to the 118 retained for analysis (results not shown). Based on the 90-day window following RPLT testing, 5 were classified as ASA, 13 were classified as OtherSE (including 7 cases with SI-related hospital admissions but no suicidal behavior; 2 with aborted/interrupted suicide attempts, and 4 with preparatory behavior), and the remaining 100 were classified as noSE. (Note that a few additional ASA and OtherSE occurred during the one-year follow-up, but fell outside the 90 day window after an RPLT testing session.) Detailed results for each outcome group are summarized in Table 2.

Table 2

Table 2. Behavioral results and RL model variables, by 90-day outcome group.

3.3 RPLT and prediction of suicide attempts

Compared to the noSE reference group, percent optimal responding on reward-based trials was higher in the ASA group (Figure 2A; OR=1.08 [1.03-1.12], p<.001), but not the OtherSE group (OR=0.98 [0.96-1.01], p=.138). Percent optimal responding on punishment-based trials was not different from noSE in either the ASA group (Figure 2B; OR=0.95 [0.89-1.01], p=.087) or the OtherSE group (OR=1.02 [0.97-1.08], p=.420).

Figure 2

Figure 2. Percent optimal responding to (A) reward-based stimuli and (B) punishment-based stimuli on the RPLT, across the 118 data files that could be associated with 90-day prospective outcomes. Note each participant could contribute RPLT data from multiple testing sessions. ASA, actual suicide attempt; OtherSE, other suicide-related event, excluding ASA; noSE, no ASA or other SE. Plots created using ggplot2 (59) for R; red lines indicate group mean and dots represent individual data points. The GEE analysis, which takes into account the repeated-measures structure of the data (not depicted in this figure) indicated increased reward-based learning was significantly predictive of upcoming ASA (OR=1.08, 95% CI=1.03-1.12, p<.001) but not OtherSE (OR=0.98, 95%CI=0.96-1.01, p=.138), compared to the noSE reference group.

Our primary hypothesis was that RPLT behavior could predict upcoming ASA, above and beyond the contributions of other standard risk variables. Table 3 summarizes the results of the GEE model using RPLT variables as predictors, adjusting for suicide risk variables of total lifetime ASAs (assessed at baseline) and SSI score (assessed at time of RPLT testing). Odds of ASA increased with greater number of prior ASAs and higher percent optimal responding on reward-based trials. No parameters emerged as significant predictors of OtherSE.

Table 3

Table 3. Results of GEE model, predicting actual suicide attempt (ASA) or other suicidal event excluding ASA (OtherSE) within 90 days, based on RPLT behavioral variables.

3.4 Reinforcement learning model

The RL model was run on the 118 data files, and parameter estimates derived for each (Table 2). As shown in Figure 3, whereas the noSE group tended to show R₀>0 (i.e., valuing the neutral/ambiguous outcome as similar to a mild reward), the ASA group tended to show R₀<0 (i.e., valuing the neutral ambiguous outcome as similar to a mild punisher) (OR 0.44 [0.22-0.89], p=.022). Values of R₀ in the OtherSE group did not differ from the noSE reference group (OR 1.42 [0.56-3.64], p=.460). No other RL model parameters emerged as significant predictors of ASA or OtherSE (all p>.250, except β which approached significance as a predictor of ASA, OR 0.09 [0.01-1.30], p=.078).

Figure 3

Figure 3. Mean parameter estimates in the reinforcement learning (RL) model, by 90-day outcome group. (A, B) LR+/LR- = learning rate on trials with better- or worse-than-expected outcomes; (C) β = explore/exploit parameter (higher values = more tendency to explore); (D) R₀ = subjective value of ambiguous/neutral feedback (R₀>0 equivalent to reward; R₀<0 equivalent to punishment); other abbreviations as in Figure 2. Note each participant could contribute data from 1+ testing session. Plots created using ggplot2 (59) for R; red lines indicate group mean and dots represent individual data points. The GEE analysis, which takes into account the repeated-measures structure of the data (not depicted in this figure) indicated that lower R₀ was significantly predictive of upcoming ASA (OR=0.44, 95% CI=0.22-0.89, p=.022) but not OtherSE (OR=1.42, 95% CI=0.56-3.64, p=.460), compared to the noSE reference group.

To determine whether R₀ could help predict upcoming ASA, above and beyond the contributions of other standard risk variables, Table 4 summarizes the results of a GEE using R₀ as a predictor, adjusting for lifetime ASAs and contemporaneous SSI. Odds of an ASA increased as number of prior ASAs increased and as R₀ decreased.

Table 4

Table 4. Results of GEE model, predicting actual suicide attempt (ASA) or other suicidal event excluding ASA (OtherSE) within 90 days based on estimated parameters from reinforcement learning (RL) model.

4 Discussion

Cognitive function in general, and decision-making in particular, have long been recognized as part of the clinical profile of individuals who have attempted suicide (53). An important feature of decision-making is how we learn to adapt our responses based on feedback, such as rewarding and punishing outcomes. However, important questions remain regarding whether feedback-based learning is altered in suicidality, and if so whether it may prospectively predict whom among a group of at-risk individuals are likely to attempt suicide within an upcoming, near-term time window. The current study used a task that interleaved reward-based and punishment-based trials, and showed that better learning to obtain reward was associated with increased risk for suicide attempt within the next 90 days, and remained predictive even after controlling for other clinical variables (e.g., current suicidal ideation severity and history of prior attempts).

At first glance, it might appear paradoxical to observe better reward-learning performance in the ASA group. However, enhanced performance on reward-based trials of the RPLT could reflect a number of possible underlying cognitive mechanisms. Computational modeling, using the RL model, was used to examine latent cognitive parameters and revealed a key finding related to the no-feedback outcome (R₀): specifically, upcoming ASA was associated with negative valuation of R₀, and this provides an interpretation of the facilitated reward-based learning on the RPLT. We discuss these points next.

4.1 Subjective valuation of ambiguous outcomes, in those with upcoming ASA

Because the RPLT interleaves reward-based and punishment-based trials, R₀ is ambiguous: on a reward-based trial, R₀ represents failure to obtain reward, but on a punishment-based trial, R₀ represents successful avoidance of punishment. In fact, if the reward-based and punishment-based trials are trained in separate blocks, R₀ tends to be negatively-valued in the reward-based trials and positively-valued in the punishment-based trials (37). When both trial types are intermixed, different individuals can vary in their subjective valuation of R₀, but among non-psychiatric populations, R₀ is typically valued as equivalent to a mild reinforcer, equivalent to a “glass half full” interpretation (“At least I wasn’t punished!”) (37, 54). In the current study, the noSE and OtherSE outcomes were similarly associated with a mildly positive valuation of the no-feedback outcome (R₀>0). In contrast, upcoming ASA was associated with a negative valuation of the no-feedback outcome (R₀<0), comparable to a mild punisher: a “glass half empty” interpretation (“I failed to get rewarded.”).

Importantly, this provides a possible mechanism that could underlie the facilitated reward-based learning observed on the RPLT in the ASA group. Specifically, valuation of R₀<0 (i.e., negative valuation of ambiguous outcome) would tend to facilitate reward learning by increasing the contrast between outcomes of explicit reward vs. non-reward, compared to groups where R₀ is valued similarly to explicit reward. By the same token, R₀<0 could conceivably impair punishment learning, decreasing the contrast between explicit punishment and non-punishment, and in fact Figure 2B suggests a slight (non-significant) decrease in optimal responding on punishment trials in the ASA group, compared to the noSE and OtherSE groups. Therefore, the association of ASA with better learning from reward trials on the RPLT could reflect an underlying bias towards negative valuation of neutral outcomes. One possibility is that R₀ therefore functions as a mediator of the relationship between reward learning and ASA. In the current dataset, R₀ was itself estimated based on the reward- and punishment-based learning during RPLT, but future studies could examine R₀ (derived from RPLT) as a possible mediator of reward learning on other tasks.

Beyond the RPLT task itself, these findings on negative evaluation of ambiguous feedback have implications for adaptation processes in daily life experienced by individuals at near-term risk of suicide. There may be a pointed deficit in learning from situational contingencies that do not provide explicit feedback. For example, one may respond adaptively to a stressor, yet not encounter a clear signal of “reward,” which may lead one to abandon that adaptive response. In general, given that the majority of daily life consists of events that are more ambiguous than an explicit reward or explicit punishment, a tendency to view all such ambiguous events as mildly punishing could promote feelings of helplessness and hopelessness – which in turn increase risk for suicidality. These results and interpretation broadly parallel the well-established finding of negative bias in depression (55) and of biases in social cognition in borderline personality disorder (56), which is a disorder characterized by high rates of suicidal behavior.

4.2 Relationship to prior research on feedback-based learning

We are aware of three other studies that applied an RL model to feedback-based learning in suicidality. The first two studies, by Dombrovski and colleagues (11, 17), used a simple forced-choice probabilistic task in which participants receive positive (rewarding) feedback following correct choices and negative (punishing) feedback following incorrect choices; those studies reported no differences between individuals with a history of prior suicide attempt compared to control groups on initial learning, in terms of either behavior or RL model variables, although the attempters were impaired at flexibly updating their response rules when contingencies were reversed. A third study, by Dixon-Gordon and colleagues (20) considered a probabilistic learning task in patients at high- or low-risk for suicide, and found no group differences on either learning accuracy or RL model variables, although group differences did emerge on the task after negative emotion induction. It remains an interesting open question whether similar results might emerge using the RPLT after a reversal and/or after negative emotion induction. However, the lack of group differences on basic response acquisition in these prior studies likely reflects both the simpler task designs and the fact that there were no ambiguous or neutral outcomes. It appears that the RPLT task design, with its frequent ambiguous feedback, allowed negative subjective interpretations of this ambiguous feedback to drive better reward-based learning in the ASA group.

In contrast to the current finding in patients at risk for suicide, several studies with the RPLT and other feedback-based learning tasks have found that depressive disorders tend to be associated with decreased learning from reward (25, 26) and/or increased learning from punishment (57). While depression and suicidality often co-occur and likely interact, the pattern of impaired learning from reward observed in depressed patients is therefore opposite from the enhanced reward learning found in the ASA group in the current study; further, as shown in the Appendix (Supplementary Figure 10), depression was associated with increased LR+ in the current sample. It is important to note that, while depression is typically associated with decreased reward processing, it has also been shown that antidepressant medication moderates RPLT behavior (25), and it is likely that many of the patients in the current sample were on antidepressant medications, complicating interpretation. Despite this, MDD status – while positively associated with both ASA and OtherSE outcomes (see Table 1) – did not significantly improve prediction of these outcomes compared to RPLT variables alone (see Appendix: Supplementary Table 6).

Another prior study found a pattern of improved reward learning and reduced R₀ in Veterans with severe symptoms of PTSD compared to peers with few/no PTSD symptoms (21); this pattern is qualitatively similar to, although less extreme than, the results associated with ASA outcomes in the current study. However, in the current study, PTSD rates were actually somewhat lower in the ASA than OtherSE and noSE groups (Appendix: Supplementary Figure 11), and PTSD itself did not emerge as a predictor of either ASA or OtherSE (Appendix: Supplementary Table 7). Unfortunately, the earlier PTSD study did not collect information regarding suicidal behavior; nevertheless, it appears that PTSD alone does not adequately explain the results of the current study. Rather, it appears that upcoming ASA is associated with a unique profile of negative valuation of ambiguous outcomes. An important question for further research is whether this valuation is relatively fixed for one individual or may wax and wane over time (i.e., in the days or weeks leading up to an ASA vs. after the attempt is made). Additionally, altered reward learning may be a specific mechanism contributing to the increased risk for suicide attempt in those with PTSD, and should be explored in future studies.

Finally, in understanding the contrast between current findings and previous studies, it should also be emphasized that the earlier studies were retrospective, for example, comparing individuals with a history of prior suicide attempt against participants with no history of suicidal thoughts or behaviors (11, 17). In contrast, the current study prospectively followed participants already considered at high risk for suicide and who often had a history of one or more prior suicide attempts. It is therefore possible that reward-based learning among the ASA group in the current sample would appear impaired if contrasted to performance in a group of healthy volunteers with no prior history of suicidal thoughts or behaviors.

4.3 Limitations

There are several important limitations of this study. The first is that it is based on a small sample of Veterans, all of whom were characterized as high-risk; in fact, prior-year suicide-related hospital admission was an entry criterion for the study. Thus, it should not be assumed that the results obtained here would generalize to a broader population.

More importantly, even in this high-risk sample, there were a small number of ASAs and OtherSEs within the one-year follow-up period. The low base rate for suicide events, even in a high-risk sample, has been a key challenge for both clinical and basic research studies of suicide, and speaks to the need to replicate findings in additional samples.

Additionally, two ASAs occurred in our sample that could not be included in our prospective analysis, in one case because the ASA occurred more than 90 days after an RPLT session; in the other case, one individual had two separate ASAs within the 90-day follow-up window after an RPLT testing session, and so only the first one was counted. The current study was explicitly designed to have short 90-day follow-up periods, given that risk prediction will be most useful clinically with a narrow prediction window. In the interest of increasing the number of ASA outcomes, some prior studies have considered a longer follow-up window (e.g., 180 days (51, 52)), which of course allows more ASAs to be captured; however, even in those prior studies, the rate of ASA hovers around 10-15% over six months.

Although it remains true that the key findings in this study are driven by a small number of ASAs which occurred in a low number of participants, an important feature of the current study was repeated measurements across time, which allowed at least some initial examination of whether task performance and parameters are likely to change within-subjects, and thus to predict the timing of ASA. While extremely preliminary (based on only 3 patients), supplemental analysis (Appendix: Supplementary Figure 9) suggests that the same individuals show patterns indicative of upcoming ASA when tested on the RPLT shortly before ASA, but perform more like the noSE group when no ASA will occur in the next few months. If this can be validated in a larger sample, this would provide important information that could greatly aid targeting of clinical resources not just to identify individuals who are generally at high risk for suicide, but rather allowing these resources to be targeted in precise time windows of particularly high risk.

Given all of the above, the current results must be replicated in a larger dataset, and ideally in a sample large enough to validate in the predictive relationship in a hold-out group, as well as with frequent within-subject measurements to confirm whether facilitated reward learning (and negatively valued R₀) may be state indicators that indicate periods of upcoming risk, rather than trait characteristics in a high-risk subgroup.

There are also limitations related to the sample itself. The focus on a Veteran sample meant that females were underrepresented, which limited power to detect possible sex differences. Prior studies with larger and more balanced samples have not to our knowledge reported sex effects on RPLT or related tasks (37, 54); however, sex differences in decision-making could well occur within the context of patients at risk for suicide. Emerging research also suggests a link between traumatic brain injury (TBI) and suicidality; however, we could not examine this in our current sample due to the absence of TBI cases in the ASA outcome group, likely due to small sample size (however, see Appendix: Supplementary Figure 13).

The current study also experienced challenges with follow-up assessments, including the interruption of in-person research testing during the COVID-19 pandemic; we were able to obtain follow-up outcome information on most participants enrolled before the shut-down, and only a few participants were censored due to death or withdrawal; of the withdrawn participants, one was by the participant’s request and the others were withdrawn by the research team due to participant incarceration or justice involvement, which is another common challenge in this population. There were also two participants whose RPLT data were excluded as non-compliant. There were no obvious differences in the censored or noncompliant subjects, compared to the rest of the sample, but differences might be detected in a larger sample. It is also important to recognize that no individuals died by suicide within the follow-up period; while most participants (78%) had a history of prior suicide attempt, the cognitive profile of those who die by suicide may be markedly different from those with severe suicidal ideation and/or suicidal behaviors who do not ultimately die by suicide.

Additionally, in the current sample there was insufficient information available regarding psychotropic medication use. Antidepressant medications have previously been shown to affect performance on the RPLT in patients with depressive disorder (25), specifically by reducing performance on punishment-based learning. Given the high rates of depression in the current sample, it remains unclear whether antidepressant use may have influenced the results, and if so whether this interaction differs in those with upcoming ASA or OtherSE. Performance on the RPLT and related tasks is also affected in patients with Parkinson’s disease on vs. off dopaminergic medication (24, 58) and in patients with current opiate addiction (22), further confirming the ability of psychoactive substances to affect how individuals process and respond to rewarding and punishing feedback. Important questions remain regarding whether similar effects of serotonergic, dopaminergic, opioid, and other medications would be evidenced in patients at risk for suicide. We do note, however, that in this sample, there did not appear to be differences related to lifetime abuse of opiates (see Appendix: Supplementary Figure 12).

5 Conclusion

Despite these limitations, the results suggest an intriguing feature of cognition that appears to be associated with, and predictive of, upcoming near-term ASA in individuals considered at risk for suicide. Given that much of everyday life is neither explicitly rewarding nor explicitly punishing, the vast majority of human experience may be comparable to ambiguous or neutral reinforcement; if these events are viewed negatively (R₀<0), this could have real-world implications, particularly in understanding individuals’ negative expectations and interpretations of neutral situations. In fact, a focus on reducing negative bias has been the hallmark of evidence-based interventions such as Cognitive Behavioral Therapy for suicidality and depression. Feedback-based learning tasks, with concomitant modeling to extract latent parameters, may therefore also assist in objective assessment of these cognitive biases, and their potential modification by therapy, in those at risk for suicide.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: A deidentified dataset supporting this work, along with the behavioral task software, modeling code, and scripts for statistical analysis, are posted on Open Science Framework (OSF) at https://osf.io/p328a/.

Ethics statement

The studies involving humans were approved by VA New Jersey Health Care System Institutional Review Board (IRB) and VA Central IRB Panel 3. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

CM: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Software, Writing – original draft, Writing – review & editing. RP: Formal analysis, Investigation, Writing – original draft, Writing – review & editing. VR: Formal analysis, Project administration, Writing – review & editing. CD: Formal analysis, Writing – review & editing. JK: Conceptualization, Writing – review & editing. AK: Investigation, Project administration, Writing – review & editing. KR: Project administration, Writing – review & editing. LH: Investigation, Writing – review & editing. RM: Investigation, Writing – review & editing. AI: Conceptualization, Formal analysis, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was supported by the U.S. Department of Veterans Affairs (VA) through Clinical Sciences Research and Development Service (CSR&D) Merit Award #I01 CX001826. The contents do not represent the views of the U.S. Department of Veterans Affairs or the United States Government.

Acknowledgments

For assistance with recruitment, screening and data collection, the authors wish to thank Miriam Latorre, Sarah Mann, Michael Callaghan, David Siegel, Megan Sedita, Jill DelPozzo, Esther Bennett, Carla Mastroianni, and Petty Tineo. For assistance with data management, the authors wish to thank Rosette Epstein, and Victoria Stiritz. For permission to use the NOUN images for task stimuli, the authors wish to thank Michael C. Hout and Jessica Horst.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1492332/full#supplementary-material

References

1. Garnett MF, Curtin SC. Suicide Mortality in the United States, 2001-2021. NCHS Data Brief, No 464 Hyattsville, MD: National Center for Health Statistics (2023).

Google Scholar

2. Fox KR, Huang X, Guzmán EM, Funsch KM, Cha CB, Ribeiro JD, et al. Interventions for suicide and self-injury: A meta-analysis of randomized controlled trials across nearly 50 years of research. Psychol Bull. (2020) 146:1117–45. doi: 10.1037/bul0000305

PubMed Abstract | Crossref Full Text | Google Scholar

3. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: A meta-analysis of 50 years of research. Psychol Bull. (2017) 143:187–232. doi: 10.1037/bul0000084

PubMed Abstract | Crossref Full Text | Google Scholar

4. Glenn CR, Nock MK. Improving the short-term prediction of suicidal behavior. Am J Prev Med. (2014) 47:S176–80. doi: 10.1016/j.amepre.2014.06.004

PubMed Abstract | Crossref Full Text | Google Scholar

5. Bagge CL, Littlefield AK, Wiegand TJ, Hawkins E, Trim RS, Schumacher JA, et al. A controlled examination of acute warning signs for suicide attempts among hospitalized patients. Psychol Med. (2023) 53:2768–76. doi: 10.1017/S0033291721004712

PubMed Abstract | Crossref Full Text | Google Scholar

6. Gordon JA, Avenevoli S, Pearson JL. Suicide prevention research priorities in health care. JAMA Psychiatry. (2020) 77:885. doi: 10.1001/jamapsychiatry.2020.1042

PubMed Abstract | Crossref Full Text | Google Scholar

7. Sutton RS, Barto AG. Reinforcement Learning An Introduction second edition. 2nd edition. Cambridge, MA: MIT Press (2018).

Google Scholar

8. Hélie S, Shamloo F, Novak K, Foti D. The roles of valuation and reward processing in cognitive function and psychiatric disorders. Ann N Y Acad Sci. (2017) 1395:33–48. doi: 10.1111/nyas.13327

PubMed Abstract | Crossref Full Text | Google Scholar

9. Lissek S, van Meurs B. Learning models of PTSD: Theoretical accounts and psychobiological evidence. Int J Psychophysiol. (2015) 98:594–605. doi: 10.1016/j.ijpsycho.2014.11.006

PubMed Abstract | Crossref Full Text | Google Scholar

10. Pike AC, Robinson OJ. Reinforcement learning in patients with mood and anxiety disorders vs control individuals: A systematic review and meta-analysis. JAMA Psychiatry. (2022) 79:313–22. doi: 10.1001/jamapsychiatry.2022.0051

PubMed Abstract | Crossref Full Text | Google Scholar

11. Dombrovski AY, Clark L, Siegle GJ, Butters MA, Ichikawa N, Sahakian BJ, et al. Reward/Punishment reversal learning in older suicide attempters. Am J Psychiatry. (2010) 167:699–707. doi: 10.1176/appi.ajp.2009.09030407

PubMed Abstract | Crossref Full Text | Google Scholar

12. Bettis AH, Benningfield MM, Dao A, Dickey L, Pegg S, Venanzi L, et al. Self-injurious thoughts and behaviors and alterations in positive valence systems: A systematic review of the literature. J Psychiatr Res. (2022) 156:579–93. doi: 10.1016/j.jpsychires.2022.10.033

PubMed Abstract | Crossref Full Text | Google Scholar

13. Millner AJ, Robinaugh DJ, Nock MK. Advancing the understanding of suicide: the need for formal theory and rigorous descriptive research. Trends Cognit Sci. (2020) 24:704–16. doi: 10.1016/j.tics.2020.06.007

PubMed Abstract | Crossref Full Text | Google Scholar

14. Montague PR, Dolan RJ, Friston KJ, Dayan P. Computational psychiatry. Trends Cognit Sci. (2012) 16:72–80. doi: 10.1016/j.tics.2011.11.018

PubMed Abstract | Crossref Full Text | Google Scholar

15. Dayan P. Dopamine, reinforcement learning, and addiction. Pharmacopsychiatry. (2009) 42:S56–65. doi: 10.1055/s-0028-1124107

PubMed Abstract | Crossref Full Text | Google Scholar

16. Averbeck B, O’Doherty JP. Reinforcement-learning in fronto-striatal circuits. Neuropsychopharmacology. (2022) 47:147–62. doi: 10.1038/s41386-021-01108-0

PubMed Abstract | Crossref Full Text | Google Scholar

17. Dombrovski AY, Szanto K, Clark L, Reynolds CF, Siegle GJ. Reward signals, attempted suicide, and impulsivity in late-life depression. JAMA Psychiatry. (2013) 70:1020–30. doi: 10.1001/jamapsychiatry.2013.75

PubMed Abstract | Crossref Full Text | Google Scholar

18. Daw ND. Trial-by-trial data analysis using computational models. In: Delgado MR, Phelps EA, Robbins EA, editors. Decision making, affect, and learning: Attention and performance. Oxford University Press, Oxford, UK (2009). p. 1–26.

Google Scholar

19. Huys QJM, Maia TV, Frank MJ. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci. (2016) 19:404–13. doi: 10.1038/nn.4238

PubMed Abstract | Crossref Full Text | Google Scholar

20. Dixon-Gordon KL, Waite EE, Ammerman BA, Haliczer LA, Boudreaux ED, Rathlev N, et al. Learning from gain and loss: Links to suicide risk. J Psychiatr Res. (2022) 147:126–34. doi: 10.1016/j.jpsychires.2021.12.016

PubMed Abstract | Crossref Full Text | Google Scholar

21. Myers CE, Moustafa AA, Sheynin J, VanMeenen KM, Gilbertson MW, Orr SP, et al. Learning to obtain reward, but not avoid punishment, is affected by presence of PTSD symptoms in male veterans: empirical data and computational model. PloS One. (2013) 8:e72508. doi: 10.1371/journal.pone.0072508

PubMed Abstract | Crossref Full Text | Google Scholar

22. Myers CE, Sheynin J, Balsdon T, Luzardo A, Beck KD, Hogarth L, et al. Probabilistic reward- and punishment-based learning in opioid addiction: Experimental and computational data. Behav Brain Res. (2016) 296:240–8. doi: 10.1016/j.bbr.2015.09.018

PubMed Abstract | Crossref Full Text | Google Scholar

23. Kéri S, Moustafa AA, Myers CE, Benedek G, Gluck MA. {alpha}-Synuclein gene duplication impairs reward learning. Proc Natl Acad Sci U.S.A. (2010) 107:15992–4. doi: 10.1073/pnas.1006068107

PubMed Abstract | Crossref Full Text | Google Scholar

24. Bódi N, Kéri S, Nagy H, Moustafa A, Myers CE, Daw N, et al. Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson’s patients. Brain. (2009) 132:2385–95. doi: 10.1093/brain/awp094

PubMed Abstract | Crossref Full Text | Google Scholar

25. Herzallah MM, Moustafa AA, Natsheh JY, Abdellatif SM, Taha MB, Tayem YI, et al. Learning from negative feedback in patients with major depressive disorder is attenuated by SSRI antidepressants. Front Integr Neurosci. (2013) 7:67. doi: 10.3389/fnint.2013.00067

PubMed Abstract | Crossref Full Text | Google Scholar

26. Herzallah MM, Khdour HY, Taha AB, Elmashala AM, Mousa HN, Taha MB, et al. Depression reduces accuracy while parkinsonism slows response time for processing positive feedback in patients with Parkinson’s disease with comorbid major depressive disorder tested on a probabilistic category-learning task. Front Psychiatry. (2017) 8:84. doi: 10.3389/fpsyt.2017.00084

PubMed Abstract | Crossref Full Text | Google Scholar

27. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: Initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry. (2011) 168:1266–77.

PubMed Abstract | Google Scholar

28. Beck AT, Kovacs M, Weissman A. Assessment of suicidal intention: the Scale for Suicide Ideation. J Consult Clin Psychol. (1979) 47:343–52. doi: 10.1037//0022-006x.47.2.343

PubMed Abstract | Crossref Full Text | Google Scholar

29. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc. (2005) 53:695–9. doi: 10.1111/j.1532-5415.2005.53221.x

PubMed Abstract | Crossref Full Text | Google Scholar

30. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. (1998) 59:22–3.

PubMed Abstract | Google Scholar

31. Schwab KA, Baker G, Ivins B, Sluss-Tiller M, Lux W, Warden D. The Brief Traumatic Brain Injury Screen (BTBIS): Investigating the validity of a self-report instrument for detecting traumatic brain injury in troops returning from deployment in Afghanistan and Iraq. Neurology. (2006) 66:A235.

Google Scholar

32. Keane TM, Fairbank JA, Caddell JM, Zimering RT, Taylor KL, Mora CA. Clinical evaluation of a measure to assess combat exposure. psychol Assess. (1989) 1:53–5. doi: 10.1037/1040-3590.1.1.53

Crossref Full Text | Google Scholar

33. Beck AT, Steer RA, Ball R, Ranieri W. Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. J Pers Assess. (1996) 67:588–97. doi: 10.1207/s15327752jpa6703_13

PubMed Abstract | Crossref Full Text | Google Scholar

34. Beck AT, Weissman A, Lester D, Trexler L. The measurement of pessimism: the hopelessness scale. J Consult Clin Psychol. (1974) 42:861–5. doi: 10.1037/h0037562

PubMed Abstract | Crossref Full Text | Google Scholar

35. Peirce JW. Generating stimuli for neuroscience using PsychoPy. Front Neuroinform. (2009) 2:10.2008. doi: 10.3389/neuro.11.010.2008

PubMed Abstract | Crossref Full Text | Google Scholar

36. Horst JS, Hout MC. The Novel Object and Unusual Name (NOUN) Database: A collection of novel images for use in experimental research. Behav Res Methods. (2016) 48:1393–409. doi: 10.3758/s13428-015-0647-3

PubMed Abstract | Crossref Full Text | Google Scholar

37. Moustafa AA, Gluck MA, Herzallah MM, Myers CE. The influence of trial order on learning from reward vs. punishment in a probabilistic categorization task: Experimental and computational analyses. Front Behav Neurosci. (2015) 9:153. doi: 10.3389/fnbeh.2015.00153

PubMed Abstract | Crossref Full Text | Google Scholar

38. Watkins CJCH, Dayan P. Q-learning. Mach Learn. (1992) 8:279–92. doi: 10.1007/BF00992698

Crossref Full Text | Google Scholar

39. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE. Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci (USA). (2007) 104:16311–6.

PubMed Abstract | Google Scholar

40. Myers CE, Del Pozzo J, Perskaudas R, Dave CV, Chesin MS, Keilp JG, et al. Impairment in recognition memory may be associated with near-term risk for suicide attempt in a high-risk sample. J Affect Disord. (2024) 350:7–15. doi: 10.1016/j.jad.2024.01.018

PubMed Abstract | Crossref Full Text | Google Scholar

41. Myers CE, Dave CV, Callahan M, Chesin MS, Keilp JG, Beck KD, et al. Improving the prospective prediction of a near-term suicide attempt in veterans at risk for suicide, using a go/no-go task. Psychol Med. (2023) 53:4245–54. doi: 10.1017/S0033291722001003

PubMed Abstract | Crossref Full Text | Google Scholar

42. Interian A, Chesin M, Kline A, Miller R, St. Hill L, Latorre M, et al. Use of the Columbia-suicide severity rating scale (C-SSRS) to classify suicidal behaviors. Arch Suicide Res. (2018) 22:278–94. doi: 10.1080/13811118.2017.1334610

PubMed Abstract | Crossref Full Text | Google Scholar

43. Liang K-Y, Zeger SL. Longitudinal analysis using generalized linear models. Biometrika. (1986) 73:13–22. doi: 10.1093/biomet/73.1.13

Crossref Full Text | Google Scholar

44. Zeger SL, Liang KY, Albert PS. Models for longitudinal data: A generalized estimating equation approach. Biometrics. (1988) 44:1049–60. doi: 10.2307/2531734

PubMed Abstract | Crossref Full Text | Google Scholar

45. Shiffman S. Conceptualizing analyses of ecological momentary assessment data. Nicotine Tobacco Res. (2014) 16:S76–87. doi: 10.1093/ntr/ntt195

PubMed Abstract | Crossref Full Text | Google Scholar

46. Ballinger GA. Using generalized estimating equations for longitudinal data analysis. Organ Res Methods. (2004) 7:127–50. doi: 10.1177/1094428104263672

Crossref Full Text | Google Scholar

47. Ma Y, Mazumdar M, Memtsoudis SG. Beyond repeated-measures analysis of variance: Advanced statistical methods for the analysis of longitudinal data in anesthesia research. Reg Anesth Pain Med. (2012) 37:99–105. doi: 10.1097/AAP.0b013e31823ebc74

PubMed Abstract | Crossref Full Text | Google Scholar

48. Schober P, Vetter TR. Repeated measures designs and analysis of longitudinal data: If at first you do not succeed-try, try again. Anesth Analg. (2018) 127:569–75. doi: 10.1213/ANE.0000000000003511

PubMed Abstract | Crossref Full Text | Google Scholar

49. Touloumis A. R package multgee: A generalized estimating equations solver for multinomial responses. J Stat Softw. (2015) 64:1–14. doi: 10.18637/jss.v064.i08

Crossref Full Text | Google Scholar

50. R Core Team. R: A language and environment for statistical computing. In: R Foundation for Statistical Computing. Vienna, Austria (2023). Available at: https://www.R-project.org (Accessed January 06, 2025).

Google Scholar

51. Cha CB, Najmi S, Park JM, Finn CT, Nock MK. Attentional bias toward suicide-related stimuli predicts suicidal behavior. J Abnorm Psychol. (2010) 119:616–22. doi: 10.1037/a0019710

PubMed Abstract | Crossref Full Text | Google Scholar

52. Nock MK, Park JM, Finn CT, Deliberto TL, Dour HJ, Banaji MR. Measuring the suicidal mind: implicit cognition predicts suicidal behavior. Psychol Sci. (2010) 21:511–7. doi: 10.1177/0956797610364762

PubMed Abstract | Crossref Full Text | Google Scholar

53. Dombrovski AY, Hallquist MN. The decision neuroscience perspective on suicidal behavior: Evidence and hypotheses. Curr Opin Psychiatry. (2017) 30:7–14. doi: 10.1097/YCO.0000000000000297

PubMed Abstract | Crossref Full Text | Google Scholar

54. Moustafa AA, Sheynin J, Myers CE. The role of informative and ambiguous feedback in avoidance behavior: Empirical and computational findings. PloS One. (2015) 10:e0144083. doi: 10.1371/journal.pone.0144083

PubMed Abstract | Crossref Full Text | Google Scholar

55. Everaert J, Podina IR, Koster EHW. A comprehensive meta-analysis of interpretation biases in depression. Clin Psychol Rev. (2017) 58:33–48. doi: 10.1016/j.cpr.2017.09.005

PubMed Abstract | Crossref Full Text | Google Scholar

56. Lazarus SA, Cheavens JS, Festa F, Zachary Rosenthal M. Interpersonal functioning in borderline personality disorder: a systematic review of behavioral and laboratory-based assessments. Clin Psychol Rev. (2014) 34:193–205. doi: 10.1016/j.cpr.2014.01.007

PubMed Abstract | Crossref Full Text | Google Scholar

57. Reinen JM, Whitton AE, Pizzagalli DA, Slifstein M, Abi-Dargham A, McGrath PJ, et al. Differential reinforcement learning responses to positive and negative information in unmedicated individuals with depression. Eur Neuropsychopharmacol. (2021) 53:89–100. doi: 10.1016/j.euroneuro.2021.08.002

PubMed Abstract | Crossref Full Text | Google Scholar

58. Frank MJ, Seeberger LC, O’reilly RC. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. (2004) 306:1940–3. doi: 10.1126/science.1102941

PubMed Abstract | Crossref Full Text | Google Scholar

59. Wickham H. ggplot2: Elegant graphics for data analysis. New York: Springer-Verlag (2016).

Google Scholar