Prediction and Prevention in the Clinical High-Risk for Psychosis Paradigm: A Review of the Current Status and Recommendations for Future Directions of Inquiry

Prediction and prevention of negative clinical and functional outcomes represent the two primary objectives of research conducted within the clinical high-risk for psychosis (CHR-P) paradigm. Several multivariable “risk calculator” models have been developed to predict the likelihood of developing psychosis, although these models have not been translated to clinical use. Overall, less progress has been made in developing effective interventions. In this paper, we review the existing literature on both prediction and prevention in the CHR-P paradigm and, primarily, outline ways in which expanding and combining these paths of inquiry could lead to a greater improvement in individual outcomes for those most at risk.

Introduction

The ability to detect, predict, and delay or prevent the onset of psychotic disorders has been the focus of the clinical high-risk for psychosis (CHR-P) research paradigm for the past 25 years (1). Approximately 20–25% of at-risk individuals develop psychosis within a 2–3-year study period. The prospective nature of the CHR-P paradigm (2) is conducive to building prognostic models (3) to enable more accurate prediction of psychosis (4, 5). A set of such predictive algorithms is now available (6–10), but none has thus far been widely implemented clinically (11). Significantly less progress has been made in developing interventions that delay or prevent psychosis transition (12) and in tailoring treatment selection based on individual risk profiles. Here, we critically review the status of research on prediction of outcomes and prevention of psychosis. We argue that a better understanding of heterogeneous outcomes of the at-risk state could improve efforts to develop effective and personalized interventions (13). We outline concrete directions for this new scope of inquiry, including incorporating prognostic stratification into intervention studies, considering an array of clinical outcomes (e.g., remission, social and functioning outcomes, and treatment outcomes), validating these outcomes with clinical and biological markers, and implementing state-of-the-art methods within these respective lines of research toward the goal of understanding, predicting, and intervening at the level of the individual.

Predicting Clinical Outcomes

Prediction of Psychosis Transition

Most of the work within the CHR-P paradigm to date has focused on building models to predict and individual's likelihood of transition to psychosis. A handful of externally-validated multivariable prediction algorithms—or “risk calculators”—are now available (11, 14). These models, largely comprised of clinical, demographic, and neurocognitive measures, have the potential for assessment in a single outpatient clinic visit.

In the first iteration of the North American Prodrome Longitudinal Study (NAPLS) (6), an empirically derived model identified additional risk factors beyond the CHR syndrome criteria to predict psychosis transition, including genetic risk for schizophrenia with recent functional decline, unusual thought content, and either suspiciousness and paranoia or impaired social functioning. These additional factors increased the positive predictive power from 35 to 74–81% (4). In NAPLS2, predictor variables were selected from the literature for inclusion in an individualized risk calculator for determining risk of psychosis transition (15). These variables included several from the NAPLS1 model plus verbal learning and memory performance, speed of processing, age, stressful life events, and history of trauma. This risk calculator ultimately performed well, achieving a Harrel's C-index of 0.71 (15). Importantly, this model was externally validated in the Early Detection, Intervention, and Prevention of Psychosis Program (EDIPPP) (16) sample (C-index = 0.79) (17) and was later also later validated in the Shanghai at Risk for Psychosis study (SHARP) (10) sample (Area Under the Curve [AUC] = 0.63) (18). This latter validation step showed that while this risk calculator model has potential for eventual broad use in samples outside of the U.S., there may still be significant differences in, for example, a Chinese population which consists of different social and cultural backgrounds that are potentially not captured by a model derived in an American sample.

Outside of the U.S., the Melbourne-based Personal Assessment and Crisis Evaluation (PACE) study (9) developed a classification algorithm based on symptom information (accuracy = 64.6%) (19), which has not been externally validated in an independent sample. Although robust internal cross-validation techniques were used to test generalizability, this approach does not provide sufficient evidence for generalizability given that the model learns iteratively from all of the data. The Shanghai-based SHARP study identified individual SIPS items to include in a prediction model (SIPS-RC). These items overlapped with previous models and included functional decline, unusual thought content, suspiciousness, social anhedonia, expression of emotion, ideational richness and dysphoric mood (AUC = 0.80) (20). When externally validated alongside the NAPLS2 risk calculator in a small (n = 68) North American SIPS-based CHR sample, the SIPS-RC and NAPLS2 risk calculator performed with fair and moderate accuracy, respectively, in discriminating between converters and non-converters (SIPS-RC AUC = 0.65, NAPLS2 risk calculator AUC = 0.71) (21). For a more detailed description of each of these predictive algorithms, please refer to Worthington, Cao and Cannon (14).

While external validation has shown promising initial results for these risk algorithms, performance in primary care settings or secondary mental health care settings (e.g., outside of highly specialized CHR-P samples) has yet to be thoroughly tested. A recent study evaluated the performance of the NAPLS2 risk calculator in the European-based Personalized Prognostic Tools for Early Psychosis Management study (PRONIA, www.pronia.eu), which includes individuals experiencing recent-onset depression (ROD) in addition to psychosis risk syndromes (22). After significant calibration across samples to account for cross-consortium differences in demographics and symptom severity, the original NAPLS2 model performed best when validated in the broader risk sample (e.g., CHR-P/ROD), demonstrating the potential prognostic utility of psychosis risk signatures in broader risk populations.

Identifying psychosis in non-risk enriched mental health care settings requires the consideration of comorbid diagnoses from more widely-used instruments such as the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) (23) or the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) (24–26). One such risk calculator was developed using a sample from the South London and the Maudsley (SLaM) National Health Service Foundation Trust services (n = 91,199). Psychosis risk syndrome criteria were necessary but insufficient in predicting the development of psychotic disorders in this secondary mental health care setting. Beyond the CHR-P criteria, ICD-10 diagnoses of acute and transient psychotic disorders and bipolar disorder predicted psychosis onset with fairly high accuracy when externally validated [C-index = 0.79 (27) and C-index = 0.73 (28)]. When validated in a comparable US-based sample (n > 2,000,000), the model maintained a good level of accuracy in detecting eventual psychosis onset (C-index = 0.67) (29). A significant challenge to this specific replication included differences in help-seeking behaviors and recruitment strategies between the UK-based discovery sample and the US-based replication sample. Accounting for differences in healthcare culture and economics will be discussed more in depth in a latter portion of this paper.

Prediction of Functional Outcomes

Levels of global, social and role functioning are consistently evaluated alongside attenuated psychotic symptoms (APS) as ecological markers of personal and occupational impairment. However, level of functioning is infrequently considered as a primary outcome of interest, potentially due to the lack of diagnostic outcome associated with functioning as compared to transition to psychotic illness, which represents a putative disorder. Further, studies vary in the subtype of functioning that is assessed—global functioning as measured by the one-item Global Assessment of Functioning (GAF) scale (30), and social or role functioning as measured by the Global Functioning Social scale (GFS), the Global Functioning Role scale (GFR) (31), and the Social and Occupational Functioning Assessment Scale (SOFAS) (32)—and often focus on the level of functioning that is most impacted in a given sample, which may also potentially dilute the strength and consistency of findings regarding “functioning” in the CHR-P syndrome.

While functioning may improve over time for non-converters, this group often remains impaired as compared to nonpsychiatric healthy controls (33). Among non-converters, remitters have shown improved social functioning, but role functioning remained impaired for all CHR groups as compared to healthy controls (34). Non-remitting non-converters (e.g., “sustainers”) continued to show significant functioning impairments at 6 years of follow-up, suggesting that functioning deficits in this population are enduring and may require specialized interventions (35). Although several “risk calculators” exist to predict psychosis transition, only two comparable multivariable models have been developed to predict functioning outcomes, despite potentially enduring functioning deficits exhibited in non-converters that may require specialized interventions (34, 35). One limitation of developing such models is that it is necessary to either predict functioning on a continuous scale, which may not translate to clear clinical decisions, or to determine cutoffs signifying positive or negative outcomes, which collapses much of the nuance captured in the continuous scales. Nevertheless, it remains important to understand an individual's likelihood for achieving positive or negative functional outcomes independent of the progression of APS.

The PRONIA study team developed accurate and generalizable models to predict social and role functioning at 1 year of follow-up (36). A model that incorporated both clinical and structural neuroimaging data outperformed clinical raters' estimations of social functional outcomes (combined model AUC = 0.86; clinical rater AUC = 0.72). Models predicting role functioning, however, did not outperform clinical raters' estimations. Role functioning may be more sensitive to temporal fluctuations of environmental and clinical factors, creating an additional challenge in producing reliable predictive models of this particular outcome (36, 37). The PACE study team also developed a model predicting functioning, which performed similarly to their model predicting psychosis transition (accuracy = 62.5%) (19). In this model, attention disturbances, asocial anhedonia and thought disorder were most predictive of negative functional outcomes. It is worth noting that neither of these algorithms have yet been replicated in external samples.

Prediction of Remission Outcomes

Remission from a CHR-P syndrome is an important and relatively unexplored outcome. In NAPLS2, trajectory modeling of non-converters showed that a third remitted in symptoms and functioning (38). Predicting CHR-P remission at the level of the individual could inform treatment decisions by allocating more intensive treatments to those most likely to experience negative outcomes than for those more likely to naturally recover (“remitters”). Understanding protective factors associated with remission could also help to inform and tailor interventions for the CHR-P population as remission may serve as a more proximal treatment outcome for intervention studies as compared to psychosis prevention. Further, remitters are a confound in randomized clinical trials (RCTs) wherein the true efficacy of an intervention may not be detected if a significant portion of participants naturally remit, potentially contributing to an inflated placebo effect (39).

While criteria for both symptom and functional remission have been established (35, 40), the clinical validity associated with the timing and duration of remission has not been adequately studied. An individual may meet remission criteria at only a single study visit and relapse at the next or may remain remitted throughout the remainder of the study—both would be considered “remitters” even though the actual clinical picture may vary considerably. This lack of clarity in remission outcome timing contributes to discrepancies in understanding recovery rates. Between 24 and 51% of non-converters may exhibit symptom remission during follow-up while only 24–28% demonstrated full clinical and functional recovery (33, 34, 41, 42). It is also unknown whether remission represents a risk factor for later relapse of APS (or even eventual psychosis transition)—an outcome that can only be studied with very long follow-up periods.

Individual clinical predictors of remission may include higher neurocognitive functioning at baseline, higher levels of social and/or role functioning at baseline, and lower levels of APS at baseline (34, 43, 44). Only one study has developed a generalizable multivariable model predicting remission as an outcome variable. This study took a data-driven approach to selecting relevant features and building a model predicting remission in the NAPLS3 sample and performed well when tested in an independent validation sample (AUC = 0.66), exhibiting performance comparable to models predicting conversion to psychosis (45).

Gaps and Future Directions in Prediction

Although progress has been made in developing generalizable risk models for multiple outcomes from the CHR-P syndrome, there is still room for improvement before prediction algorithms enter the clinic. Thorough external validation strategies have been limited to date. External testing in diverse risk cohorts and varied health care settings is necessary to: (1) account for cross-site and cross-country sample differences; (2) ensure compatibility across clinical instruments; (3) test the risk signatures measured by each algorithm; and (4) ensure the specificity of eventual psychosis onset as compared to negative outcomes more broadly (26, 46, 47). Methodological differences in both feature selection (e.g., data-driven and theory-driven) and model development (e.g., Cox regression and machine learning classifiers) provide novel avenues of model discovery but limit the possibility of direct comparisons of performance between models developed in heterogeneous samples (11).

Levels of patient distress and pathways of ascertainment differ across cohorts and pose an additional challenge to the development of prediction models. The studies described here rely on either intensive outreach to the general population (self-referral), relationships with established healthcare providers (clinician-referral), or a combination thereof. These approaches yield samples with different levels of baseline conversion risk. Self-referrals result in a less enriched risk sample and a lower eventual rate of psychosis transition (48). This heterogeneity in recruitment strategies may contribute to a reduction in performance when replicating models in other samples.

Further, the two main frameworks used to describe at-risk populations—the Criteria of Prodromal Syndromes (COPS) (49) and the Comprehensive Assessment of At-Risk Mental State (CAARMS) (50)—may be more or less amenable to these recruitment strategies. As the CAARMS permits for a broader range and duration of APS, a broader recruitment strategy may be warranted. In NAPLS2 (COPS criteria), the sample is younger, has a higher severity of APS at baseline, and experiences a higher psychosis transition rate than the PRONIA study (CAARMS criteria) (22). The CAARMS criteria also tends to be used in countries with socialized medicine (e.g., PRONIA in Europe, the PACE clinic in Australia) whereas the COPS criteria tend to be used in the US, which operates through a privatized healthcare system. Milder cases with more general symptoms may be detected earlier using the CAARMS criteria in countries with significantly reduced barriers to specialized healthcare.

Another notable difference across studies includes the domains of clinical information from which predictor variables are sampled for a given model. Symptom information and certain demographic variables are the most commonly sampled domains included in the models described above, and neurocognitive measures have been included in some, but not all, models. Beyond these domains, other important health-related factors such as substance use (51, 52), sleep patterns (53), and history of traumatic brain injury (54) have been studied in CHR-P samples but rarely included in multivariate prediction models and could differ across cohorts. A more systematic approach across studies to selecting predictors from different variable domains and ensuring the availability of data from these domains could reduce variability in this area as well. In particular, it should be noted that exposure and adherence to pharmacological or psychosocial interventions at baseline may have an association with later transition to psychosis (55, 56); however, medication exposure, medication adherence, and participation in therapy are rarely included in prognostic models. While some descriptive studies have shown that eventual converters and non-converters show no difference in baseline antipsychotic exposure (57), emerging trends suggest that antipsychotic use may predict negative outcomes that may be more pronounced in milder CHR-P cases (57, 58). As antipsychotic medications are typically administered for more severe symptom presentations (2), and administration is difficult to control for in naturalistic studies, more systematic research is needed to clarify and establish the relationships between severity, treatment utilization, and eventual clinical outcomes through prediction studies.

Variance in ascertainment and sample enrichment could be addressed through a clinical staging model involving sample stratification based on the severity of symptom clusters (59). More general and less severe symptoms are categorized as a lower risk stage whereas more severe prodromal symptoms are categorized at a higher risk stage, and interventions would be allocated based on this stratification. More research is needed to validate such a staging model, as baseline symptoms may not predicate linear progression of the psychosis risk syndrome. Poor outcomes could include psychosis transition or worsening symptoms, but may also include the development of other forms of psychopathology. The ability to measure these outcomes will depend on the risk enrichment within the sample, resulting from the ascertainment strategies implemented. To validate prediction models across heterogeneous samples, patient distress, and/or method of ascertainment should be included as a co-variate and future studies should explore the relationships between distress, ascertainment method and level of risk for psychosis.

In examining the most predictive variables included in the prediction models described (e.g., suspiciousness and unusual thought content), it is worth considering the potentially tautological relationship with the definition of psychosis transition (e.g., the presence of psychotic levels of positive symptoms). The question arises of whether the at-risk state is explained by severity of specific positive symptoms which precede eventual transition to psychosis, or whether a more complex risk profile—which may include these symptoms—best explains the at-risk syndrome. In other fields of biomedicine, risk syndromes signal a state during which intervention could successfully delay or prevent the onset of disease. Mild Cognitive Impairment (MCI) is a condition between normal cognitive functioning and impaired cognitive functioning which typically precedes Alzheimer's disease and instantiates the risk state for eventual severe cognitive decline. With proper assessment, early identification and intervention may slow the development of Alzheimer's disease from this precursor state (60). In the clinical high-risk state, positive symptoms seem to be specific to psychosis (2) and thus signal an important opportunity for more intensive assessment and/or intervention regardless of the presence of other risk factors. Nevertheless, the interaction of positive symptoms with other risk factors suggest that these symptoms alone may not be sufficiently predictive of either conversion to psychosis or remission (15), thus reaffirming the ongoing effort to build informative multivariate models that predict heterogeneous outcomes from the CHR-P state that could inform the optimal intervention for a given risk profile.

Another significant methodological challenge in validating and implementing risk models is the necessity of converting a continuous output value to a binary classification output. In the models described, the output is a score ranging from 0.0 to 1.0 indicating the likelihood an individual will experience the outcome of interest (e.g., psychosis transition). The optimal score cutoff that correctly classifies the eventual outcome may differ across studies. In NAPLS2, a score of 0.2 indicated that conversion occurred at a higher rate than non-conversion above the 0.2 predicted risk level (15); however, when tested in the PRONIA sample, the 0.2 cutoff did not yield the same results, and accounting for between-cohort differences in average demographics and symptom severity were necessary to achieve robust external validation at the 0.2 cutpoint (22). These adjustments pose a barrier to clinical implementation and the assessment of new help-seeking individuals and further inquiry into how to calibrate across diverse risk samples will be an essential step toward the clinical implementation of prediction models.

Beyond clinical information, several biomarkers have also been considered for inclusion in the prediction models described. The role of biomarkers in prediction has been reviewed elsewhere (61); thus, we will focus on the additive value of biomarkers to existing clinical models. In this context, no added biomarkers have been shown to replicate across models or studies. In the PRONIA study, a combined clinical and neuroimaging model outperformed either modality alone in predicting social functioning outcomes. This finding, although not yet replicated, suggests that specific configurations of clinical and biological information may achieve the best predictive accuracy as compared to either method independently, consistent with findings that both neuroanatomical and behavioral changes are associated with psychosis transition (62).

Two studies have examined the potential additive role of biomarkers to the NAPLS2 risk calculator. The first added a measure of deviance in neuroanatomical maturity (or “brain age”), previously shown to predict psychosis transition in 12–17 year-olds (63). When added to the risk calculator, however, brain age completely overlapped with chronological age (64), itself a significant predictor of conversion in this sample, with younger cases showing a higher risk. Another study added baseline cortisol to the NAPLS2 risk calculator and found that this measure did contribute significantly to the model's ability to discriminate between eventual converters and non-converters (65). This finding (also not yet replicated) suggests that this measure of baseline cortisol may capture a meaningful biological pattern not otherwise captured through clinical measures.

Further considerations for the inclusion of biomarkers relate to the portability to clinical practice. A major advantage of clinical prediction models is that clinical assessments are inexpensive, non-invasive, and scores can typically be determined within a single day, if not immediately. Thus, clinical decisions could be made quickly. Administering biological tests typically involves a higher cost, longer time to administer or obtain results (e.g., assaying blood or saliva samples, processing MRI scan results), and more invasive or uncomfortable procedures. To optimize clinical utility, biomarkers should contribute significantly to the predictive accuracy of clinical risk models in order to outweigh the higher costs in time, money, and subject burden inherent in obtaining these measures.

As psychosis transition has been the primary outcome of interest in the CHR-P research paradigm, the most substantial progress has been made in predicting this outcome. Other outcomes of interest have received less attention in the field until recently, but warrant further exploration, as it has been understood for some time that the CHR-P population comprises heterogeneous clinical presentations and outcomes (66). The future of risk prediction in the CHR-P paradigm might include the use of a set of algorithms to predict an array of outcomes and the predicted time course of each of these outcomes. This ambitious goal will be discussed more in depth in a latter portion of this paper.

Prevention

Complementary to the goal of predicting clinical outcomes is the goal of delaying or preventing psychosis through early interventions. A recent review found that only 20 randomized controlled trials (RCTs) testing the effectiveness of either psychosocial (10 of these trials), pharmacological interventions (7 trials), or both (3 trials) had been completed specifically for CHR-P individuals (67). Even within this small number of trials, modalities of treatment and measured outcomes varied greatly. While psychosis transition is a primary outcome, other outcomes include change in positive symptoms, change in negative symptoms, levels of functioning, quality of life and other outcomes related to psychopathology, such as depression and anxiety.

Cognitive behavioral therapy (CBT) has been studied more than other psychosocial modalities (e.g., family interventions, cognitive remediation, and integrative psychotherapy), but has yielded mixed results in either preventing the onset of psychosis or reducing APS. While some studies demonstrated a reduction in rates of psychosis transition (68, 69) or APS reduction (70), others showed no effect of CBT (71) or a larger effect of another modality (e.g., directive listening) in reducing symptom distress (72). Of the other psychosocial modalities, family-focused therapy (FFT) reduced symptoms but not transition rates (73); integrative psychological therapy decreased transition rates at 12 and 24 months (74); and cognitive remediation had no effect on symptoms (75) but improved verbal memory, processing speed and social adjustment (76, 77). Of the pharmacological therapies tested, large trials of omega-3 fatty acids showed no effect in reducing transition rates and symptoms as compared to placebo (78, 79) despite promising earlier studies to this effect (80). Trials of the N-methyl-D-aspartate receptor modulators D-serine and glycine showed no effect in either reducing transition rates or APS (81, 82). Trials with antipsychotics have also been mixed. Antipsychotic medications alone improved positive symptoms but not transition rates (83, 84) and antipsychotics in combination with psychosocial therapy had a more significant effect (85, 86).

Importantly, recent meta-analyses showed that, integrated across studies, no psychosocial or pharmacological interventions significantly reduce APS or transition rates, despite promising results from individual trials (87–90). Further, limitations inherent in both pairwise and network meta-analytic methods render the results of these studies even less conclusive due to comparing unequal control groups across studies, the small number of overall intervention studies, and unequal comparisons across SIPS-based studies and CAARMS-based studies (67). In addition, more recent intervention studies were powered based on higher rates of psychosis transition observed in early CHR-P studies that have since declined, thus resulting in studies that were ultimately underpowered (67).

Gaps and Future Directions in Prevention

A significant gap in prevention work to date is that the heterogeneity within CHR-P samples has been largely ignored, potentially contributing to the mixed and null results described (12). One way to overcome this is to incorporate some measure of an individual's outcome likelihood into clinical trial design and analysis. A recent study incorporated predicted risk of conversion (15) into re-examining the efficacy of FFT and found that higher risk individuals showed greater APS improvement with the FFT intervention as compared to the control group, but lower risk individuals showed no difference in APS improvement between the two intervention groups (91). A concrete step toward improving our understanding of effective interventions would be to similarly re-examine results of existing trials by incorporating conversion risk as either a measure of stratification or as a covariate in predicting the trial outcome. Another step toward improving the precision of clinical trials would be to incorporate baseline risk assessment and subsequent stratification into the initial design and randomization scheme. These two recommendations represent a crucial way in which the interdigitation of prediction and prevention could broaden the field's scope of inquiry and improve outcomes.

Another way to address outcome heterogeneity is to develop a core outcomes assessment set (COS) for clinical trials. The primary target of interventions would be expanded beyond psychosis transition to include change in positive, negative, anxiety, or depression symptoms, functioning, neurocognition, and recovery or remission (92). Determining a set of outcomes that were included in all trials would facilitate direct comparisons of interventions across trials (93). As prediction models start to extend beyond measuring conversion to psychosis as the primary outcome of interest, so should target outcomes of clinical trials. Whether it is possible to prevent the development of frank psychosis is unknown; however, interventions could likely ameliorate APS and comorbid symptoms of anxiety and depression or improve functioning. Given that the majority of CHR-P individuals do not convert to psychosis yet continue to manifest moderate levels of symptoms and functional impairments (33, 41, 94), targeting non-conversion outcomes is essential to alleviating impairment for this group.

Future Directions of Inquiry

Ultimately, joining the two tasks of prediction and prevention more systematically could significantly increase the utility and precision of findings from the CHR-P paradigm. Understanding an individual's probable trajectory is most helpful when there is a chance to intervene and potentially improve this trajectory. Approaching this goal could be relatively straightforward [e.g., adding risk scores to clinical trial outcome analysis or stratification (91)], although expanding the scope of inquiry and methods used to interrogate relevant questions will help support the effort to leverage predictive analytics toward improving outcomes.

Improving Prediction Algorithms for Treatment Selection

Increasingly sophisticated machine learning classification algorithms, such as random forest, support vector machines, Bayesian classifiers, etc., have been used in the CHR population (and psychiatry more broadly) to predict individual outcomes (95). Multi-site consortium studies (e.g., NAPLS, PRONIA, PSYSCAN [http://psyscan.eu]) produce large sample sizes and the opportunity to develop generalizable and robust prediction models with external validation procedures. In these large samples, which are most conducive to implementing machine learning techniques, certain guidelines should inform the implementation of these methods (96). Beyond sample size, the number and type of predictor variables included should be carefully considered. Too many predictors could lead to over-fitting, instability and poor generalization of a model. Thus, feature selection should be performed prior to model fitting either through a data-driven approach, a theory-driven approach, or a hybrid of these two approaches which could leverage the strengths of each (97). The number of features to retain for model development is another key consideration; in general, it is recommended that the ratio of predictors to outcome instances should be approximately 10:1 (98).

A major obstacle faced for prediction in the CHR-P population is the low base rate of conversion to psychosis (99). Machine learning algorithms are biased toward predicting the majority outcome class and accuracy alone cannot provide sufficient information about discrimination between true positive and true negative cases. An algorithm predicting remission in 20/100 CHR-P individuals that is 80% accurate could fail to correctly detect any remission cases but still appear to perform well. Thus, metrics such as sensitivity, specificity, balanced accuracy, and area under the curve (AUC) should also be included. Further, balanced sampling techniques should be considered when severely imbalanced classes threaten the generalizability of a model. One such approach is the synthetic minority oversampling technique (SMOTE) which creates a new data set by oversampling the minority outcome class based on information from neighboring data points, thus reducing bias induced during model learning (100). Balanced sampling approaches should only be used in discovery samples and models should not be validated in synthetically balanced samples. Implementing these sampling approaches has shown to increase model performance in predicting remission (45) and should be considered as a tool to improve prediction in the CHR-P population.

Biomarkers and neuroimaging data may be able to improve prediction algorithms that face a potential limitation in achieving the highest level of predictive accuracy with clinical information alone (61). The inclusion of neurobiological data has not yet yielded these results; however, there is some progress in this direction (36, 65). In a controlled simulation, it was shown that environmental, clinical, neuroimaging and blood biomarker assessments were sequentially examined as testing stages, wherein each subsequent modality added to the positive predictive values (PPV) not detected from the previous modality (59). This type of staged or dynamic approach to risk assessment could greatly increase our ability to predict outcomes, but may become cumbersome in clinical implementation. If overwhelming evidence suggests that the inclusion of a biomarker reliably adds value to individual prediction, it should be included in a prognostic assessment. A decision then arises of whether the field should focus on improving prognostic accuracy by adding biomarkers, or on mobilizing existing prediction efforts with more pragmatic clinical models to inform the selection and development of interventions to improve outcomes more proximally.

A method that warrants more in-depth study involves updating baseline prediction models with clinical information from follow-up visits, called joint modeling (101). This method has been applied in two CHR-P samples and showed significantly better performance than baseline clinical models alone (102, 103). Very few studies have approached risk prediction with heterogeneous clinical profiles and developmental trajectories in mind (33, 38). Ascertaining an individual's probable outcome from a baseline assessment fails to account for variability in syndrome history (e.g., duration of prodromal symptoms prior to initial assessment), and the progression, deterioration, or improvement of symptoms after baseline. Although the baseline “snapshot” approach has been useful to date, a framework that continuously updates based on new or changing clinical information for an individual has the potential to significantly improve outcome prediction.

A related gap in our current research paradigm that warrants expansion is the study of more secondary outcomes such as functioning, remission, and treatment response. This is an area where biomarkers could also help to validate clinical constructs that have not been fully pinned down. Within the example of remission, described previously, the lack of agreement on the time course, duration, and clinical picture of remission could be clarified through careful examination of how symptoms and biomarker signatures co-evolve over time. A very long follow-up period (e.g., > 2 years) would be necessary to understand the time course of remission onset and duration, how remission relates to recovery and relapse, whether neurobiological patterns map onto these clinical presentations, and what would predispose an individual to spontaneous remission as opposed to intervention-driven remission. Similar questions could guide our understanding of outcomes such as social and role functioning as well as neurocognition.

Improving Treatments and Treatment Selection With Predictive Analytics

The existing literature on psychosocial interventions for the CHR-P population is inconclusive; however, predictive analytics could help account for heterogeneity and inform the type and intensity of recommended services. Biomarkers could potentially play a role in these decisions and more robust inquiry could reveal which mechanisms (e.g., inflammation, neuroplasticity, neurocognition) should be targeted with psychosocial or pharmacological interventions. At this point, mechanisms of these interventions for the CHR-P syndrome are not understood well-enough to match components of an individual's clinical profile to components of effective interventions; however, it would be valuable to have this level of understanding to inform treatment selection.

As no biomarker has consistently replicated in studies assessing individual risk of conversion, the role of clinical predictors is again emphasized. Given that psychosis and its risk syndrome are assessed at the clinical level, symptom-level measurements should also summarize underlying neurobiological changes. With precise clinical measurements, the brain-behavior link is manifested in the individual's responses. Nevertheless, there is room to refine clinical instruments to maximize clinical utility and portability which may also increase consistency in measurements used across consortium studies. Recently, NAPLS investigators developed an abbreviated version of the SIPS—the Mini-SIPS—which could minimize the time and patient burden required to arrive at a relevant risk syndrome diagnosis (104). This measure is yet to be validated although this and similar initiatives could greatly increase the accessibility and transportability of risk assessment measurements in both specialized high-risk clinics and broader healthcare settings.

Conclusions

Risk prediction and intervention are currently separate but related goals of the clinical high risk for psychosis research paradigm. Progress has been made in ascertaining risk and some progress has been made in identifying effective interventions; however, very little work has combined these tasks—assessing the likelihood of experiencing heterogeneous clinical outcomes and selecting treatment accordingly—to ultimately improve outcomes. The continuation of large multi-site consortium studies will help to facilitate the following ongoing goals: developing and validating risk prediction algorithms for a range of clinical outcomes; identifying time-dependent biomarker signatures that validate clinical trajectories; accounting for sample heterogeneity in clinical trial design; and matching individual clinical profiles to specific mechanisms of effective interventions. Although there is still a lot of work to be done, the future of the CHR-P paradigm holds great promise to significantly improve outcomes for those at the highest levels of risk.

Author Contributions

MW and TC conceptualized this review. MW conducted the review and drafted the manuscript. TC provided critical revision of the manuscript. All authors approved the final version to be published.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Yung AR, McGorry PD. The prodromal phase of first-episode psychosis: past and current conceptualizations. Schizophr Bull. (1996) 22:353–70. doi: 10.1093/schbul/22.2.353

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Sykes LAY, Ferrara M, Addington J, Bearden CE, Cadenhead KS, Cannon TD, et al. Predictive validity of conversion from the clinical high risk syndrome to frank psychosis. Schizophr Res. (2020) 216:184–91. doi: 10.1016/j.schres.2019.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Fusar-Poli P, Borgwardt S, Bechdolf A, Addington J, Riecher-Rössler A, Schultze-Lutter F, et al. The psychosis high-risk state: a comprehensive state-of-the-art review. JAMA Psychiatry. (2013) 70:107–20. doi: 10.1001/jamapsychiatry.2013.269

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Cannon TD, Cadenhead K, Cornblatt B, Woods SW, Addington J, Walker E, et al. Prediction of psychosis in youth at high clinical risk: a multisite longitudinal study in North America. Arch Gen Psychiatry. (2008) 65:28–37. doi: 10.1001/archgenpsychiatry.2007.3

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Fusar-Poli P, Bonoldi I, Yung AR, Borgwardt S, Kempton MJ, Valmaggia L, et al. Predicting psychosis: meta-analysis of transition outcomes in individuals at high clinical risk. Arch Gen Psychiatry Mar. (2012) 69:220–9. doi: 10.1001/archgenpsychiatry.2011.1472

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Addington J, Cadenhead KS, Cannon TD, Cornblatt B, McGlashan TH, Perkins DO, et al. North American Prodrome Longitudinal Study: a collaborative multisite approach to prodromal schizophrenia research. Schizophr Bull. (2007) 33:665–72. doi: 10.1093/schbul/sbl075

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Addington J, Cadenhead KS, Cornblatt BA, Mathalon DH, McGlashan TH, Perkins DO, et al. North American prodrome longitudinal study (NAPLS 2): overview and recruitment. Schizophr Res. (2012) 142:77–82. doi: 10.1016/j.schres.2012.09.012

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Addington J, Liu L, Brummitt K, Bearden CE, Cadenhead KS, Cornblatt BA, et al. North American Prodrome Longitudinal Study (NAPLS 3): Methods and baseline description. Schizophr Res. (2020) 1–6. doi: 10.1016/j.schres.2020.04.010

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Phillips LJ, Leicester SB, O'Dwyer LE, Francey SM, Koutsogiannis J, Abdel-Baki A, et al. The PACE Clinic: identification and management of young people at “ultra” high risk of psychosis. J Psychiatr Pract. (2002) 8:255–69. doi: 10.1097/00131746-200209000-00002

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Zhang T, Li H, Woodberry KA, Seidman LJ, Zheng L, Li H, et al. Prodromal psychosis detection in a counseling center population in China: an epidemiological and clinical study. Schizophr Res. (2014) 152:391–9. doi: 10.1016/j.schres.2013.11.039

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Sanfelici R, Dwyer DB, Antonucci LA, Koutsouleris N. Individualized diagnostic and prognostic models for patients with psychosis risk syndromes: a meta-analytic view on the state-of-the-art. Biol Psychiatry. (2020) 88:349–60. doi: 10.1016/j.biopsych.2020.02.009

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Fusar-Poli P, de Pablo GS, Correll CU, Meyer-Lindenberg A, Millan MJ, Borgwardt S, et al. Prevention of psychosis: advances in detection, prognosis, and intervention. JAMA Psychiatry. (2020) 77:755–65. doi: 10.1001/jamapsychiatry.2019.4779

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Thompson E, Millman ZB, Okuzawa N, Mittal V, DeVylder J, Skadberg T, et al. Evidence-based early interventions for individuals at clinical high risk for psychosis: a review of treatment components. J Nervous Ment Dis. (2015) 203:342–51. doi: 10.1097/NMD.0000000000000287

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Worthington MA, Cao H, Cannon TD. Discovery and validation of prediction algorithms for psychosis in youth at clinical high risk. Biol Psychiatry Cogn Neurosci Neuroimaging. (2020) 5:738–47. doi: 10.1016/j.bpsc.2019.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Cannon TD, Yu C, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, et al. An individualized risk calculator for research in prodromal psychosis. Am J Psychiatry. (2016) 173:980–8. doi: 10.1176/appi.ajp.2016.15070890

PubMed Abstract | CrossRef Full Text | Google Scholar

16. McFarlane WR, Cook WL, Downing D, Ruff A, Lynch S, Adelsheim S, et al. Early detection, intervention, and prevention of psychosis program: rationale, design, and sample description. Adolesc Psychiatry. (2012) 2:112–24. doi: 10.2174/2210676611202020112

CrossRef Full Text | Google Scholar

17. Carrión RE, Cornblatt BA, Burton CZ, Tso IF, Auther AM, Adelsheim S, et al. Personalized prediction of psychosis: external validation of the NAPLS-2 psychosis risk calculator with the EDIPPP project. Am J Psychiatry. (2016) 173:989–96. doi: 10.1176/appi.ajp.2016.15121565

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Zhang T, Li H, Tang Y, Niznikiewicz MA, Shenton ME, Keshavan MS, et al. Validating the predictive accuracy of the NAPLS-2 psychosis risk calculator in a clinical high-risk sample from the SHARP (Shanghai At Risk for Psychosis) Program. Am J Psychiatry. (2018) 175:906–8. doi: 10.1176/appi.ajp.2018.18010036

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Mechelli A, Lin A, Wood S, McGorry P, Amminger P, Tognin S, et al. Using clinical information to make individualized prognostic predictions in people at ultra high risk for psychosis. Schizophr Res. (2017) 184:32–8. doi: 10.1016/j.schres.2016.11.047

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Zhang T, Xu L, Tang Y, Li H, Tang X, Cui H, et al. Prediction of psychosis in prodrome: development and validation of a simple, personalized risk calculator. Psychol Med. (2018) 49:1990–8. doi: 10.1017/S0033291718002738

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Osborne KJ, Mittal VA. External validation and extension of the NAPLS-2 and SIPS-RC personalized risk calculators in an independent clinical high-risk sample. Psychiatry Res. (2019) 279:9–14. doi: 10.1016/j.psychres.2019.06.034

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Koutsouleris N, Worthington M, Dwyer DB, Kambeitz-Ilankovic L, Sanfelici R, Fusar-Poli P, et al. Towards generalizable and transdiagnostic tools for psychosis prediction: an independent validation and improvement of the NAPLS-2 risk calculator in the multi-site PRONIA cohort. Biol Psychiatry. (2021) 90:632–42. doi: 10.1016/j.biopsych.2021.06.023

PubMed Abstract | CrossRef Full Text | Google Scholar

23. American Psychiatric A. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®, 5th Edn. Arlington, VA: American Psychiatric Publication (2013).

Google Scholar

24. World Health Organization. International Statistical Classification of Diseases and Related Health Problems, 11th Edn (2020).

Google Scholar

25. McGorry PD, Hartmann JA, Spooner R, Nelson B. Beyond the “at risk mental state” concept: transitioning to transdiagnostic psychiatry. World Psychiatry. (2018) 17:133–42. doi: 10.1002/wps.20514

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Francesconi M, Minichino A, Carrión R, Delle Chiaie R, Bevilacqua A, Parisi M, et al. Psychosis prediction in secondary mental health services. A broad, comprehensive approach to the “at risk mental state” syndrome. Eur Psychiatry. (2017) 40:96–104. doi: 10.1016/j.eurpsy.2016.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Fusar-Poli P, Rutigliano G, Stahl D, Davies C, Bonoldi I, Reilly T, et al. Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis. JAMA Psychiatry. (2017) 74:493–500. doi: 10.1001/jamapsychiatry.2017.0284

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Fusar-Poli P, Werbeloff N, Rutigliano G, Oliver D, Davies C, Stahl D, et al. Transdiagnostic risk calculator for the automatic detection of individuals at risk and the prediction of psychosis: second replication in an independent National Health Service Trust. Schizophr Bull. (2018) 45:562–70. doi: 10.1093/schbul/sby070

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Oliver D, Wong CMJ, Bøg M, Jönsson L, Kinon BJ, Wehnert A, et al. Transdiagnostic individualized clinically-based risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: external replication in 2,430,333 US patients. Transl Psychiatry. (2020) 10:1–10. doi: 10.1038/s41398-020-01032-9

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Hall RC. Global assessment of functioning: a modified scale. Psychosomatics. (1995) 36:267–75. doi: 10.1016/S0033-3182(95)71666-8

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Cornblatt BA, Auther AM, Niendam T, Smith CW, Zinberg J, Bearden CE, et al. Preliminary findings for two new measures of social and role functioning in the prodromal phase of schizophrenia. Schizophr Bull. (2007) 33:688–702. doi: 10.1093/schbul/sbm029

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Morosini PL, Magliano L, Brambilla La, Ugolini S, Pioli R. Development, reliability and acceptability of a new version of the DSM-IV Social and Occupational Functioning Assessment Scale (SOFAS) to assess routine social funtioning. Acta Psychiatr Scand. (2000) 101:323–9. doi: 10.1111/j.1600-0447.2000.tb10933.x

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Addington J, Cornblatt BA, Cadenhead KS, Cannon TD, McGlashan TH, Perkins DO, et al. At clinical high risk for psychosis: outcome for nonconverters. Am J Psychiatry. (2011) 168:800–5. doi: 10.1176/appi.ajp.2011.10081191

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Addington J, Stowkowy J, Liu L, Cadenhead KS, Cannon TD, Cornblatt BA, et al. Clinical and functional characteristics of youth at clinical high-risk for psychosis who do not transition to psychosis. Psychol Med. (2019) 49:1670–7. doi: 10.1017/S0033291718002258

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Brandizzi M, Valmaggia L, Byrne M, Jones C, Iwegbu N, Badger S, et al. Predictors of functional outcome in individuals at high clinical risk for psychosis at six years follow-up. J Psychiatr Res. (2015) 65:115–23. doi: 10.1016/j.jpsychires.2015.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Koutsouleris N, Kambeitz-Ilankovic L, Ruhrmann S, Rosen M, Ruef A, Dwyer DB, et al. Prediction models of functional outcomes for individuals in the clinical high-risk state for psychosis or with recent-onset depression: a multimodal, multisite machine learning analysis. JAMA Psychiatry. (2018) 75:1156–72. doi: 10.1001/jamapsychiatry.2018.2165

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Cornblatt BA, Carrión RE, Addington J, Seidman L, Walker EF, Cannon TD, et al. Risk factors for psychosis: impaired social and role functioning. Schizophr Bull. (2012) 38:1247–57. doi: 10.1093/schbul/sbr136

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Allswede DM, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, Mathalon DH, et al. Characterizing covariant trajectories of individuals at clinical high risk for psychosis across symptomatic and functional domains. Am J Psychiatry. (2020) 177:164–71. doi: 10.1176/appi.ajp.2019.18111290

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Ferrarelli F, Mathalon D. The prodromal phase: time to broaden the scope beyond transition to psychosis? Schizophr Res. (2020) 216:5–6. doi: 10.1016/j.schres.2019.12.035

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Hawkins K, McGlashan T, Quinlan D, Miller T, Perkins D, Zipursky R, et al. Factorial structure of the Scale of Prodromal Symptoms. Schizophr Res. (2004) 68:339–47. doi: 10.1016/S0920-9964(03)00053-7

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Simon AE, Borgwardt S, Riecher-Rössler A, Velthorst E, de Haan L, Fusar-Poli P. Moving beyond transition outcomes: meta-analysis of remission rates in individuals at high clinical risk for psychosis. Psychiatry Res. (2013) 209:266–72. doi: 10.1016/j.psychres.2013.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Beck K, Studerus E, Andreou C, Egloff L, Leanza L, Simon AE, et al. Clinical and functional ultra-long-term outcome of patients with a clinical high risk (CHR) for psychosis. Eur Psychiatry. (2019) 62:30–7. doi: 10.1016/j.eurpsy.2019.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Lee TY, Shin YS, Shin NY, Kim SN, Jang JH, Kang D-H, et al. Neurocognitive function as a possible marker for remission from clinical high risk for psychosis. Schizophr Res. (2014) 153:48–53. doi: 10.1016/j.schres.2014.01.018

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Glenthøj LB, Kristensen TD, Wenneberg C, Hjorthøj C, Nordentoft M. Predictors of remission from the ultra-high risk state for psychosis. Early Interv Psychiatry. (2020) 15:104–12. doi: 10.1111/eip.12917

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Worthington M, Addington J, Bearden CE, Cadenhead K, Cornblatt B, Keshavan M, et al. Individualized prediction of prodromal symptom remission for youth at clinical high-risk for psychosis. Schizophr Bull. (2021). doi: 10.1093/schbul/sbab115

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Cornblatt BA, Carrión RE, Auther A, McLaughlin D, Olsen RH, John M, et al. Psychosis prevention: a modified clinical high risk perspective from the recognition and prevention (RAP) program. Am J Psychiatry. (2015) 172:986–94. doi: 10.1176/appi.ajp.2015.13121686

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Rosen M, Betz LT, Schultze-Lutter F, Chisholm K, Haidl TK, Kambeitz-Ilankovic L, et al. Towards clinical application of prediction models for transition to psychosis: a systematic review and external validation study in the PRONIA sample. Neurosci Biobehav Rev. (2021) 125:478–92. doi: 10.1016/j.neubiorev.2021.02.032

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Fusar-Poli P, Schultze-Lutter F, Cappucciati M, Rutigliano G, Bonoldi I, Stahl D, et al. The dark side of the moon: meta-analytical impact of recruitment strategies on risk enrichment in the clinical high risk state for psychosis. Schizophr Bull. (2016) 42:732–43. doi: 10.1093/schbul/sbv162

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Miller TJ, McGlashan TH, Rosen JL, Cadenhead K, Ventura J, McFarlane W, et al. Prodromal assessment with the structured interview for prodromal syndromes and the scale of prodromal symptoms: predictive validity, interrater reliability, and training to reliability. Schizophr Bull. (2003) 29:703–15. doi: 10.1093/oxfordjournals.schbul.a007040

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Yung AR, Yung AR, Pan Yuen H, Mcgorry PD, Phillips LJ, Kelly D, et al. Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry. (2005) 39:964–71. doi: 10.1080/j.1440-1614.2005.01714.x

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Addington J, Case N, Saleem MM, Auther AM, Cornblatt BA, Cadenhead KS. Substance use in clinical high risk for psychosis: a review of the literature. Early Interv Psychiatry. (2014) 8:104–12. doi: 10.1111/eip.12100

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Valmaggia L, Day F, Jones C, Bissoli S, Pugh C, Hall D, et al. Cannabis use and transition to psychosis in people at ultra-high risk. Psychol Med. (2014) 44:2503–12. doi: 10.1017/S0033291714000117

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Goines KB, LoPilato AM, Addington J, Bearden CE, Cadenhead KS, Cannon TD, et al. Sleep problems and attenuated psychotic symptoms in youth at clinical high-risk for psychosis. Psychiatry Res. (2019) 282:1–7. doi: 10.1016/j.psychres.2019.112492

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Deighton S, Buchy L, Cadenhead KS, Cannon TD, Cornblatt BA, McGlashan TH, et al. Traumatic brain injury in individuals at clinical high risk for psychosis. Schizophr Res. (2016) 174:77–81. doi: 10.1016/j.schres.2016.04.041

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Woods SW, Addington J, Bearden CE, Cadenhead KS, Cannon TD, Cornblatt BA, et al. Psychotropic medication use in youth at high risk for psychosis: comparison of baseline data from two research cohorts 1998-2005 and 2008-2011. Schizophr Res. (2013) 148:99–104. doi: 10.1016/j.schres.2013.05.019

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Addington J, Heinssen R. Prediction and prevention of psychosis in youth at clinical high risk. Annu Rev Clin Psychol. (2012) 8:269–89. doi: 10.1146/annurev-clinpsy-032511-143146

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Zhang T, Wang J, Xu L, Wei Y, Tang X, Hu Y, et al. Further evidence that antipsychotic medication does not prevent long-term psychosis in higher-risk individuals. Eur Arch Psychiatry Clin Neurosci. (2021) 1–12. doi: 10.1007/s00406-021-01331-2

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Zhang T, Xu L, Tang X, Wei Y, Hu Q, Hu Y, et al. Real-world effectiveness of antipsychotic treatment in psychosis prevention in a 3-year cohort of 517 individuals at clinical high risk from the SHARP (ShangHai At Risk for Psychosis). Aust N Z J Psychiatry. (2020) 54:696–706. doi: 10.1177/0004867420917449

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Schmidt A, Cappucciati M, Radua J, Rutigliano G, Rocchetti M, Dell'Osso L, et al. Improving prognostic accuracy in subjects at clinical high risk for psychosis: systematic review of predictive models and meta-analytical sequential testing simulation. Schizophr Bull. (2017) 43:375–88. doi: 10.1093/schbul/sbw098

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Bredesen DE, Amos EC, Canick J, Ackerley M, Raji C, Fiala M, et al. Reversal of cognitive decline in Alzheimer's disease. Aging (Albany NY). (2016) 8:1250–8. doi: 10.18632/aging.100981

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Ellis JK, Walker EF, Goldsmith DR. Selective review of neuroimaging findings in youth at clinical high risk for psychosis: on the path to biomarkers for conversion. Front Psychiatry. (2020) 11:567534. doi: 10.3389/fpsyt.2020.567534

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Cannon TD, Chung Y, He G, Sun D, Jacobson A, van Erp TGM, et al. Progressive reduction in cortical thickness as psychosis develops: a multisite longitudinal neuroimaging study of youth at elevated clinical risk. Biol Psychiatry. (2015) 77:147–57. doi: 10.1016/j.biopsych.2014.05.023

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Chung Y, Addington J, Bearden CE, Cadenhead K, Cornblatt B, Mathalon DH, et al. Use of machine learning to determine deviance in neuroanatomical maturity associated with future psychosis in youths at clinically high risk. JAMA Psychiatry. (2018) 75:960–8. doi: 10.1001/jamapsychiatry.2018.1543

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Chung Y, Addington J, Bearden CE, Cadenhead K, Cornblatt B, Mathalon DH, et al. Adding a neuroanatomical biomarker to an individualized risk calculator for psychosis: a proof-of-concept study. Schizophr Res Jun. (2019) 208:41–3. doi: 10.1016/j.schres.2019.01.026

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Worthington MA, Walker EF, Addington J, Bearden CE, Cadenhead KS, Cornblatt BA, et al. Incorporating cortisol into the NAPLS2 individualized risk calculator for prediction of psychosis. Schizophr Res. (2020) 227:95–100 doi: 10.1016/j.schres.2020.09.022

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Fusar-Poli P, Cappucciati M, Borgwardt S, Woods SW, Addington J, Nelson B, et al. Heterogeneity of psychosis risk within individuals at clinical high risk: a meta-analytical stratification. JAMA Psychiatry. (2016) 73:113–20. doi: 10.1001/jamapsychiatry.2015.2324

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Addington J, Devoe DJ, Santesteban-Echarri O. Multidisciplinary treatment for individuals at clinical high risk of developing psychosis. Curr Treat Opt Psychiatry. (2019) 6:1–6. doi: 10.1007/s40501-019-0164-6

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Morrison AP, French P, Walford L, Lewis SW, Kilcommons A, Green J, et al. Cognitive therapy for the prevention of psychosis in people at ultra-high risk: randomised controlled trial. Br J Psychiatry. (2004) 185:291–7. doi: 10.1192/bjp.185.4.291

PubMed Abstract | CrossRef Full Text | Google Scholar

69. van der Gaag M, Nieman DH, Rietdijk J, Dragt S, Ising HK, Klaassen RMC, et al. Cognitive behavioral therapy for subjects at ultrahigh risk for developing psychosis: a randomized controlled clinical trial. Schizophr Bull. (2012) 38:1180–8. doi: 10.1093/schbul/sbs105

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Morrison AP, French P, Stewart SLK, Birchwood M, Fowler D, Gumley AI, et al. Early detection and intervention evaluation for people at risk of psychosis: multisite randomised controlled trial. BMJ. (2012) 344:1756–833. doi: 10.1136/bmj.e2233

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Addington J, Epstein I, Liu L, French P, Boydell KM, Zipursky RB. A randomized controlled trial of cognitive behavioral therapy for individuals at clinical high risk of psychosis. Schizophr Res. (2011) 125:54–61. doi: 10.1016/j.schres.2010.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Stain HJ, Bucci S, Baker AL, Carr V, Emsley R, Halpin S, et al. A randomised controlled trial of cognitive behaviour therapy versus non-directive reflective listening for young people at ultra high risk of developing psychosis: the detection and evaluation of psychological therapy (DEPTh) trial. Schizophr Res. (2016) 176:212–9. doi: 10.1016/j.schres.2016.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Miklowitz DJ, O'Brien MP, Schlosser DA, Addington J, Candan KA, Marshall C, et al. Family-focused treatment for adolescents and young adults at high risk for psychosis: results of a randomized trial. J Am Acad Child Adolesc Psychiatry. (2014) 53:848–58. doi: 10.1016/j.jaac.2014.04.020

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Bechdolf A, Wagner M, Ruhrmann S, Harrigan S, Putzfeld V, Pukrop R, et al. Preventing progression to first-episode psychosis in early initial prodromal states. Br J Psychiatry. (2012) 200:22–9. doi: 10.1192/bjp.bp.109.066357

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Piskulic D, Barbato M, Liu L, Addington J. Pilot study of cognitive remediation therapy on cognition in young people at clinical high risk of psychosis. Psychiatry Res. (2015) 225:93–8. doi: 10.1016/j.psychres.2014.10.021

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Loewy R, Fisher M, Schlosser DA, Biagianti B, Stuart B, Mathalon DH, et al. Intensive auditory cognitive training improves verbal memory in adolescents and young adults at clinical high risk for psychosis. Schizophr Bull. (2016) 42:S118–S26 doi: 10.1093/schbul/sbw009

PubMed Abstract | CrossRef Full Text | Google Scholar

77. Choi J, Corcoran CM, Fiszdon JM, Stevens M, Javitt DC, Deasy M, et al. Pupillometer-based neurofeedback cognitive training to improve processing speed and social functioning in individuals at clinical high risk for psychosis. Psychiatr Rehabil J. (2017) 40:33. doi: 10.1037/prj0000217

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Cadenhead K, Addington J, Cannon T, Cornblatt B, Mathalon D, McGlashan T, et al. Omega-3 fatty acid versus placebo in a clinical high-risk sample from the North American Prodrome Longitudinal Studies (NAPLS) Consortium. Schizophr Bull. (2017) 43:S16. doi: 10.1093/schbul/sbx021.042

CrossRef Full Text | Google Scholar

79. McGorry PD, Nelson B, Markulev C, Yuen HP, Schäfer MR, Mossaheb N, et al. Effect of ω-3 polyunsaturated fatty acids in young people at ultrahigh risk for psychotic disorders: the NEURAPRO randomized clinical trial. JAMA Psychiatry. (2017) 74:19–27. doi: 10.1001/jamapsychiatry.2016.2902

PubMed Abstract | CrossRef Full Text | Google Scholar

80. Amminger GP, Schäfer MR, Papageorgiou K, Klier CM, Cotton SM, Harrigan SM, et al. Long-chain ω-3 fatty acids for indicated prevention of psychotic disorders: a randomized, placebo-controlled trial. Arch Gen Psychiatry. (2010) 67:146–54. doi: 10.1001/archgenpsychiatry.2009.192

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Kantrowitz JT, Woods SW, Petkova E, Cornblatt B, Corcoran CM, Chen H, et al. D-serine for the treatment of negative symptoms in individuals at clinical high risk of schizophrenia: a pilot, double-blind, placebo-controlled, randomised parallel group mechanistic proof-of-concept trial. Lancet Psychiatry. (2015) 2:403–12. doi: 10.1016/S2215-0366(15)00098-X

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Woods SW, Walsh BC, Hawkins KA, Miller TJ, Saksa JR, D'Souza DC, et al. Glycine treatment of the risk syndrome for psychosis: report of two pilot studies. Eur Neuropsychopharmacol. (2013) 23:931–40. doi: 10.1016/j.euroneuro.2012.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

83. McGlashan TH, Zipursky RB, Perkins D, Addington J, Miller T, Woods SW, et al. Randomized, double-blind trial of olanzapine versus placebo in patients prodromally symptomatic for psychosis. Am J Psychiatry. (2006) 163:790–9. doi: 10.1176/ajp.2006.163.5.790

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Woods S, Saksa J, Compton M, Daley M, Rajarethinam R, Graham K, et al. Effects of ziprasidone versus placebo in patients at clinical high risk for psychosis. Schizophr Bull. (2017) 43:S58. doi: 10.1093/schbul/sbx021.150

CrossRef Full Text | Google Scholar

85. McGorry PD, Yung AR, Phillips LJ, Yuen HP, Francey S, Cosgrave EM, et al. Randomized controlled trial of interventions designed to reduce the risk of progression to first-episode psychosis in a clinical sample with subthreshold symptoms. Arch Gen Psychiatry. (2002) 59:921–8. doi: 10.1001/archpsyc.59.10.921

PubMed Abstract | CrossRef Full Text | Google Scholar

86. Ruhrmann S, Bechdolf A, Kühn K-U, Wagner M, Schultze-Lutter F, Janssen B, et al. Acute effects of treatment for prodromal symptoms for people putatively in a late initial prodromal state of psychosis. Br J Psychiatry. (2007) 191:s88–s95. doi: 10.1192/bjp.191.51.s88

PubMed Abstract | CrossRef Full Text | Google Scholar

87. Devoe DJ, Farris MS, Townes P, Addington J. Interventions and transition in youth at risk of psychosis: a systematic review and meta-analyses. J Clin Psychiatry. (2020) 81:17r12053. doi: 10.4088/JCP.17r12053

PubMed Abstract | CrossRef Full Text | Google Scholar

88. Davies C, Cipriani A, Ioannidis JP, Radua J, Stahl D, Provenzani U, et al. Lack of evidence to favor specific preventive interventions in psychosis: a network meta-analysis. J World Psychiatry. (2018) 17:196–209. doi: 10.1002/wps.20526

PubMed Abstract | CrossRef Full Text | Google Scholar

89. Devoe DJ, Farris MS, Townes P, Addington J. Attenuated psychotic symptom interventions in youth at risk of psychosis: a systematic review and meta-analysis. Early Interv Psychiatry. (2019) 13:3–17. doi: 10.1111/eip.12677

PubMed Abstract | CrossRef Full Text | Google Scholar

90. Davies C, Radua J, Cipriani A, Stahl D, Provenzani U, McGuire P, et al. Efficacy and acceptability of interventions for attenuated positive psychotic symptoms in individuals at clinical high risk of psychosis: a network meta-analysis. Front Psychiatry. (2018) 9:187. doi: 10.3389/fpsyt.2018.00187

PubMed Abstract | CrossRef Full Text | Google Scholar

91. Worthington MA, Miklowitz DJ, O'Brien M, Addington J, Bearden CE, Cadenhead KS, et al. Selection for psychosocial treatment for youth at clinical high risk for psychosis based on the North American Prodrome Longitudinal Study individualized risk calculator. Early Interv Psychiatry. (2020) 15:96–103. doi: 10.1111/eip.12914

PubMed Abstract | CrossRef Full Text | Google Scholar

92. Woods SW, Mourgues-Codern CV, Powers AR 3rd. Commentary. Toward a core outcomes assessment set for clinical high risk. Schizophr Res. (2020) 227:78–80. doi: 10.1016/j.schres.2020.05.008

PubMed Abstract | CrossRef Full Text | Google Scholar

93. Macefield RC, Jacobs M, Korfage IJ, Nicklin J, Whistance RN, Brookes ST, et al. Developing core outcomes sets: methods for identifying and including patient-reported outcomes (PROs). Trials. (2014) 15:1–12. doi: 10.1186/1745-6215-15-49

PubMed Abstract | CrossRef Full Text | Google Scholar

94. Simon AE, Velthorst E, Nieman DH, Linszen D, Umbricht D, de Haan L. Ultra high-risk state for psychosis and non-transition: a systematic review. Schizophr Res. (2011) 132:8–17. doi: 10.1016/j.schres.2011.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

95. Dwyer DB, Falkai P, Koutsouleris N. Machine learning approaches for clinical psychology and psychiatry. Annu Rev Clin Psychol. (2018) 14:91–118. doi: 10.1146/annurev-clinpsy-032816-045037

PubMed Abstract | CrossRef Full Text | Google Scholar

96. Grzenda A, Kraguljac NV, McDonald WM, Nemeroff C, Torous J, Alpert JE, et al. Evaluating the machine learning literature: a primer and user's guide for psychiatrists. Am J Psychiatry. (2021) 178:715–29. doi: 10.1176/appi.ajp.2020.20030250

PubMed Abstract | CrossRef Full Text | Google Scholar

97. Elhai JD, Montag C. The compatibility of theoretical frameworks with machine learning analyses in psychological research. Curr Opin Psychol. (2020) 36:83–8. doi: 10.1016/j.copsyc.2020.05.002

PubMed Abstract | CrossRef Full Text | Google Scholar

98. Cearns M, Hahn T, Baune BT. Recommendations and future directions for supervised machine learning in psychiatry. Transl Psychiatry. (2019) 9:1–12. doi: 10.1038/s41398-019-0607-2

PubMed Abstract | CrossRef Full Text | Google Scholar

99. Polari A, Lavoie S, Yuen H-P, Amminger P, Berger G, Chen E, et al. Clinical trajectories in the ultra-high risk for psychosis population. Schizophr Res. (2018) 197:550–6. doi: 10.1016/j.schres.2018.01.022

PubMed Abstract | CrossRef Full Text | Google Scholar

100. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. SMOTE: synthetic minority over-sampling technique. J Artif Intell Res. (2002) 16:321–57. doi: 10.1613/jair.953

PubMed Abstract | CrossRef Full Text | Google Scholar

101. Yuen HP, Mackinnon A, Nelson B. A new method for analysing transition to psychosis: joint modelling of time-to-event outcome with time-dependent predictors. Int J Methods Psychiatr Res. (2018) 27:1–9. doi: 10.1002/mpr.1588

PubMed Abstract | CrossRef Full Text | Google Scholar

102. Yuen HP, Mackinnon A, Hartmann J, Amminger G, Markulev C, Lavoie S, et al. Dynamic prediction of transition to psychosis using joint modelling. Schizophr Res. (2018) 202:333–40. doi: 10.1016/j.schres.2018.07.002

PubMed Abstract | CrossRef Full Text | Google Scholar

103. Studerus E, Beck K, Fusar-Poli P, Riecher-Rössler A. Development and validation of a dynamic risk prediction model to forecast psychosis onset in patients at clinical high risk. Schizophr Bull. (2020) 46:252–60. doi: 10.1093/schbul/sbz059

PubMed Abstract | CrossRef Full Text | Google Scholar

104. Woods S, Walsh B, Cannon T. Mini-SIPS 1.0, An Abbreviated Clinical Version of the Structured Interview for Psychosis-Risk Syndromes (2020). Available online at: https://cpb-us-w2.wpmucdn.com/campuspress.yale.edu/dist/d/120/files/2020/05/Mini-SIPS-version-1-0-041920.pdf