Differences Between Self-Reported Psychotic Experiences, Clinically Relevant Psychotic Experiences, and Attenuated Psychotic Symptoms in the General Population

Purpose: Psychotic experiences in childhood (such as hearing voices or being suspicious) represent an important phenotype for early intervention. However, these experiences can be defined in several ways: self-reported psychotic experiences (SRPE) rely exclusively on the child’s report, clinically validated psychotic experiences (CRPE) are based on clinical assessment, and attenuated psychotic symptoms (APS) represents a categorization to do with clinical relevance in relation to severity. Very few studies have investigated how these distinctions impact clinical and other domains. The present study aims to compare SRPE, CRPE, and APS among children and adolescents.

Methods: This study is part of the Brazilian High-Risk Cohort Study for Psychiatric Disorders, in which 2,241 individuals aged 6–14 years provided self-ratings of 20 psychotic experiences using the Community Assessment of Psychic Experiences (CAPE). A trained psychologist conducted an interview to validate or reject reported experiences and to rate the presence of APS and affective flattening. In parallel, parents provided information about child mental health to an independent interviewer. We tested the association of mutually exclusive categories of non-validated SRPE (nSRPE), clinically validated PE below the threshold for APS (nCRPE), and APS (nSRPE = 33%, nCRPE = 11%, APS = 6%), with parents’ information about the child’s positive attributes and levels of psychopathology and psychologist assessment of blunted affect.

Results: Most associations were qualitatively similar, and there was a dose–response in the strength of associations across categories, such that APS > nCRPE > nSRPE. Experiences in all three categories were associated with female sex. nSRPE were associated with overall levels of psychopathology, but to a lesser degree than nCRPE and APS. APS and nCRPE were associated with less positive attributes, with APS more so than nCRPE. Only APS was associated with affective flattening.

Conclusions: In children and adolescents, SRPE, CRPE, and APS all index liability for psychopathology, but as clinician rated relevance increases, associations get stronger and become evident across more domains.

Introduction

Children and adolescents who present with unusual experiences, such as perceptual a bnormalities and cognitive distortions that resemble delusions and hallucinations but are below the threshold for clinical psychosis (1, 2), are at risk of mental health problems in later life (1, 3, 4). Subclinical psychotic experiences (PE) are associated with psychiatric morbidity (5–7) and may impact global functioning to a larger extent than other mental health problems (8). In particular, PE index risk of the emergence of psychotic disorders (9), parasuicidal behaviors (10), and other mental health problems like anxiety and depression (11–13).

Because unusual experiences have been associated with risk of mental health problems, the parameter “presence of unusual experiences” has been used for the selection of individuals who may be targeted for preventive interventions. Consequently, clinical trials testing interventions to promote mental health among those who present with unusual experiences are emerging (3, 4), and at least one clinical guideline already recommends that children and adolescents with experiences of transient or attenuated psychotic symptoms should be referred to a specialist mental health service or an early intervention in psychosis service (2).

Although the evidence linking unusual experiences and negative mental health outcomes is consistent and justifies a focus on individuals at risk (8), markedly different methodologies for the assessment of such experiences exist, and there is very little empirical data comparing the different methods of assessment (14–17). Across the literature on unusual experiences, studies have used a range of different definitions: “psychotic experiences,” “self-reported psychotic experiences,” “false-positive psychotic experiences,” “clinically validated psychotic experiences,” and “attenuated psychotic symptoms” (APS). Thus, PE can be assessed by self-report and/or clinical judgment, in an attempt to identify the varying degrees to which one may present with perceptions and thoughts that resemble psychosis, yielding clinically relevant PE (CRPE) and clinically non-confirmed self-reported PE (nSRPE) (18). CRPE, however, may not fulfill criterion for APS as the identification of APS requires a more complex assessment on which a trained evaluator conducts an in-depth clinical interview and, based on that, has to judge whether subclinical delusions, hallucinations, or disorganization of speech surpasses a given level that is close to but below the threshold for clinical psychosis (19). Thus, a part of those considered to have CRPE are not classified as APS (hereafter: non-APS CRPE, or simply nCRPE). Although all measures (SRPE, CRPE, and APS) describe phenotypes that pertain to the same spectrum phenotype that also includes clinical psychosis, they came from different theoretical frameworks and have been tested across a range of different settings. PE are often assessed using self-reports in the context of epidemiological surveys and show relatively high prevalence rates in the general population, particularly in children and adolescents (20). APS, on the other hand, are assessed in a clinical context, with a particular focus on help-seeking late adolescents or young adults approaching the age for the peak incidence of psychosis (21). While PE are assumed to be present to varying degrees in non-clinical populations (22), APS are typically assessed in the context of help-seeking for disorders of anxiety, depression, and drug misuse and may even index imminent conversion to psychosis (19). In other words, although both PE and APS indicate risk, PE are very common among young people (20) and can be considered epidemiologically normative to a certain degree (23). APS, on the other hand, are thought to arise in the context of early pathology and may be suitable for the identification of individuals in need of preventive interventions (21, 24). PE severity and persistence over time may represent important moderators; prospective studies showed that higher loads of SRPE and more persistent SRPE are more strongly associated with psychopathology (18). Clinical validation of SRPE was examined in a number of studies, usually to examine the predictive validity of self-report against clinical judgment (14, 16, 17, 25).

Although PE and APS are frequently studied in the literature of risk of psychosis, there is not much work examining to what degree they reflect the same underlying dimension (21, 26, 27). Children and adolescents are a particularly relevant age group for at-risk interventions. Examination of the conceptual overlap of the different measurements in children and adolescents is thus important, considering that unusual experiences may be used to enrol individuals in clinical treatment (2) and that, depending on the measure, such phenomena are not rare among youth (28). When self-report is used, 17% of children aged 9–12 years and 7.5% of adolescents aged 13–18 years report hearing voices (20), while almost all of the children endorse at least one psychotic experience when 20 different PE are investigated (29). There is a growing literature on the validity of self-report against clinical judgment for the evaluation of PE, showing they form part of the same continuum (14, 25). However, no study has tested the results of the two diagnostic procedures in a population-based sample not limited to individuals seeking help from services (15–17).

The present study explores the rates and clinical correlates of SRPE, CRPE and APS in an epidemiological, school-based sample of children and adolescents. For this purpose, 2,241 children aged 6–14 years were assessed for self-report and clinical ratings of 20 PE. Children were further classified in terms of fulfilling or not the criteria for APS, based on applying an APS criterion from the CAARMS instrument (30). Parents were interviewed independently on child demographic characteristics, mental health, and behavior. The aim was to compare the convergent and divergent validity of SRPE, CRPE, and APS. For this purpose, we tested the strength of the association between mutually exclusive categories of non-validated SRPE (nSRPE), clinically relevant PE below the threshold for APS (nCRPE), and APS, with overall levels of psychopathology, positive attributes, and blunted affect. Hypotheses were that nSRPE, nCRPE, and APS assess different levels of severity but are pertaining to the same underlying construct, which ranges from poor mental health to more severe psychopathology, including psychosis. More specifically, based on previous work in this area (14, 18, 25, 31, 32) we expected nSRPE to be associated with levels of psychopathology (positive association) and indicators of positive attributes (negative association), but at lower levels than nCRPE. In addition, we expected APS, to a greater degree than nCRPE, to be associated additionally with psychopathology such as blunted affect, based on the observation of psychosis as a multidimensional clinical psychotic syndrome (33).

Methods

Study Design and Subjects

This study is part of the baseline assessment of the Brazilian High-Risk Cohort Study for Psychiatric Disorders, a cohort study of 2,511 children and adolescents in two major cities of Brazil (São Paulo and Porto Alegre). The main aim of the study was to identify developmental trajectories that lead to psychopathology, which is why children and adolescents with increased levels of familial psychopathology were oversampled. Details about the study protocol were published in a previous methodological paper (34). The project was approved by the standing ethics committee of the University of São Paulo. All parents provided written informed consent and children provided written informed assent.

In the year 2009, biological parents of all children aged 6–12 years, registered in a convenience sample of 54 public schools, were invited to participate. Parents of 9,937 children consented to participate; these children and their first-degree relatives were screened for the presence of mental disorders using the Family History Screening (FHS) (35). FHS is a structured interview conducted by lay interviewers in which parents are asked to provide information about the presence of DSM-IV major mental symptoms in each of the biological first-degree relatives. Of the 9,937 children, 1,500 were randomly selected and invited to participate and 957 responded (hereafter: “random subsample”). Children not included in the random subsample who had screened positive for mental health problems of interest were ranked according to percentage of first-degree relatives screening positive for mental disorders. Children ranked highest were recruited first and enrolment continued until the predefined maximum of 2,511 subjects was reached. This resulted in a subsample with 1,554 children with high rates of personal and familial liability for ADHD, anxiety disorders, OCD, psychotic experiences, and learning difficulties.

The 2,511 children included were further evaluated in the subsequent year, in which lay interviewers conducted household interviews with parents, while psychologists visited schools or houses to evaluate children.

Assessments of PE and APS

The assessment of PE included the following procedures:

1. Self-report: Self-report was assessed using the Community Assessment of Psychic Experiences (CAPE) (36, 37). Because low levels of literacy were expected in our sample, trained psychologists read the questions of the CAPE verbatim to the children. In case children could not understand the questions, psychologists were instructed to repeat the questions up to three times exactly as displayed in the verbatim. If after three repetitions the child could not understand the question, psychologists were allowed to read the verbatim substituting some words by synonyms, but not to change the structure of the sentence or to give children explanations or examples. CAPE investigates 20 psychotic experiences, including perceptual abnormalities, persecutory ideation, bizarre experiences, and magical thinking (Table 2 lists PE covered by CAPE). Each of the 20 items is composed of one opening question (“yes” or “no”) and two subsequent questions, on which those who endorse the symptom in the opening question are queried about the frequency and distress caused by the experience. Coding is as follows: for frequency, 1 = “never,” 2 = “sometimes,” 3 = “frequently,” 4 = “nearly always”; for distress, 1 = “not distressing,” 2 = “a bit distressing,” 3 = “quite distressing,” 4 = “very distressing.” When the opening question is coded “no,” the frequency and distress questions are coded “0.” To assert the presence or absence of PE, items were dichotomized as 0 = “no/never/sometimes” and 1 = “frequently/nearly always” for frequency items, and distress item dichotomization was as follows: 0 = “not/a bit distressing” and 1 = “quite/very distressing.”

2. Clinical confirmation of PE: A total of 42 psychology graduates worked on this project. All psychologists were trained by clinical psychiatrists with expertise in psychosis and in the evaluation of children and adolescents. After rating the CAPE self-report of the item read verbatim, psychologists were instructed to explore the clinical significance of the reported PE, asking details and examples about each item that screened positive. They were allowed to make a free clinical interview. Psychologists also gathered information about the child’s cultural background, the explanation the child gave for the experience, compatibility with age-related fantasies, and level of conviction associated with the experience. Based on this information, psychologists made a clinical judgment differentiating clinically relevant from clinically non-relevant experiences. To set the criterion of what was clinically relevant, psychologists were instructed to judge whether or not the reported experience would raise concern about the child’s mental health status in a clinical setting. Experiences that could be considered developmentally appropriate or that could be explained by contextual factors (e.g., imagination, fantasy, sleep-related phenomena, normal reaction of fear) should be disregarded (see Supplementary Material for a list of examples of experiences disregarded by clinicians for some items of CAPE). Possible scores for clinical judgment were as follows: 1 = “unlikely to be a PE,” 2 = “a little likely,” 3 = “very likely,” 4 = “certainly a PE.” When psychologists were in doubt about which score to choose, a written report about the experience reported by the child was presented for further review by a child psychiatrist. A total of 1,216 experiences were thus transcribed and reviewed. For the categorization of children in terms of having or not having CRPE, experiences considered by psychologists unlikely or a little likely were coded 0 and experiences that were very likely or certainly a PE were coded 1.

3. Attenuated psychotic symptoms: APS were assessed using the clinical rating of perceptual abnormalities, unusual mental content, and speech organization from the Comprehensive Assessment of At-Risk Mental States (CAARMS) (30). The same clinical psychologists who evaluated PE also used ratings from CAARMS to assess APS. Although CAARMS ratings were used, psychologists did not conduct the complete interview from CAARMS. Instead, after conducting a free clinical interview to inspect CAPE symptoms, they would use additional questions from CAARMS to assess symptoms not covered by CAPE, as required. Psychiatrists who had been fully trained in the instrument, and had experience in its use, trained psychologists to use CAARMS ratings. The training session included instructional information and coding exercises. CAARMS subscales have anchored scores that are as follows: 0 = “Absent,” 1 = “Questionable,” 2 = “Mild,” 3 = “Moderate,” 4 = “Moderately severe,” 5 = “Severe,” 6 = “Severe and psychotic.” Delusions can be rated in the CAARMS on one of two subscales, that is “unusual mental content” and “non-bizarre ideation” subscales. Because the anchors for both subscales are the same and individuals that are positive in one or the other will be considered positive for APS, psychologists assessed delusions in a single rating, taking into consideration all delusions displayed by the individual. As stated in the criterion for the diagnosis of “ultra-high risk” proposed by CAARMS, APS were considered sub-threshold for psychosis with a score of 3–5 on the Unusual Thought Content/Non-bizarre ideas subscale, a score of 3–4 on the Perceptual Abnormalities subscale, or a score of 4–5 on the Disorganised Speech subscale. Although scores higher than these are considered overt psychotic symptoms in CAARMS, the 15 individuals whose symptoms were rated as overt psychotic symptoms were nevertheless included in the APS sub-threshold group so as to not invalidate the comparison with CAPE ratings, which do not exclude the upper level of severity of psychotic experiences.

4. Mutually exclusive categories of PE and APS: Individuals were categorized into four mutually exclusive groups: 1) those without any PE or APS, the reference category; 2) those with at least one PE according to self-report, but no PE according to clinical evaluation nor any APS (nSRPE); 3) those with at least one PE according to clinical evaluation, but not APS (nCRPE); and 4) fulfilling criterion for APS (APS).

Assessment of Predictors

1. Socioeconomic status: Socio economic status (SES) was obtained from household assets and education of the household head according to the Brazil Criterion for Economic Classification proposed by ANEP (38). Based on this criterion, families were scored between 0 and 46, resulting in classes A (35–46, the wealthiest) to E (0–7, the poorest). The C class is the middle class in Brazil, and in 2009, it corresponded to a per-capita mean monthly income of US$475 (39).

2. Overall levels of psychopathology: Parental ratings on two screening instruments were used to access children’s overall levels of psychopathology: the Strengths and Difficulties Questionnaire (SDQ) (40) and the Child Behaviour Checklist (CBCL) (41). Because reading proficiency is low in the Brazilian population, lay interviewers read questions for parents and took notes of their answers. The SDQ covers hyperactivity and social, emotional, and conduct problems; it allows two general scores, the “total difficulties score” and the “impact score.” The “total difficulties score” ranges from 0 to 40, higher scores representing higher levels of symptoms. The “impact score” ranges from 0 to 10, higher scores denoting higher impact of behavioral problems on daily functioning. The CBCL has 118 items that are summarized in a score that ranges from 0 to 2. It covers eight domains of psychopathology: anxious/depressed, withdrawn/depressed, somatic complaints, social problems, thought problems, attention problems, rule-breaking behavior, and aggressive behavior.

3. DSM IV diagnosis: The Development and Well Being Assessment (DAWBA) (42, 43) was used to assess children’s DSM-IV diagnoses of mental disorders. The DAWBA is a package of interviews, questionnaires, and rating techniques designed to generate ICD-10 and DSM-IV or DSM-5 psychiatric diagnoses on 2- to 17-year-olds. Although, ideally, DAWBA combines multiple sources of information, the report of youth is only possible for those who are 11 years or older as younger children tend to provide non-reliable reports (44). The DAWBA interviews can be administered either by interviewers or by computer, and interviewers only have to be trained in how to apply the interview (42, 43). In our study, lay interviewers interviewed only parents, and the diagnostic ratings were revised by a psychiatrist combining information from coded items and open questions.

4. Positive attributes: This measurement was obtained from parents’ answers on the Children Youth Strengths Inventory (YSI) from DAWBA (45), a 24-item inventory accessing positive behaviors. Examples of good behaviors covered by YSI are: being good with friends, helpful at home, and polite. Scores range from 0 to 48, higher scores indicative of more positive attributes.

5. Estimated intelligence quotient (IQ): IQ was estimated using the vocabulary and block design subtests of the Weschler Intelligence Scale for Children, 3rd edition—WISC-III (46), using the Tellegen and Briggs method (47) and Brazilian norms (48).

Statistical Analysis

Stata/SE 13.1 was used for all analyses (49). The data have a cross-level structure because assessments are clustered within both psychologists and school, the same pool of psychologists visiting all schools. Data were also clustered by site (São Paulo vs. Porto Alegre city), but because there were only two cities, a dummy for city was added to the models instead of adding an extra level to the multilevel analysis (50). Before analyzing any results, intraclass correlation coefficients of possible levels for analysis were obtained for different model structures and models were compared.

The strength of the association between different outcomes and predictors was estimated using multilevel or logistic regression models. Stata xtmixed and xtmelogit commands were used for the multilevel (cross-level) linear and logistic regression models. For all models, the dependent variables (e.g., SRPE, CRPE, and APS) and the independent variables (age, sex, SES, levels of psychopathology, positive attributes, etc.) were reversed. This was done because the previously described multilevel models are the correct method to analyze the present data, and the analysis of a categorical dependent variable in a multilevel setting is not possible in Stata. In order to avoid demographic characteristics and IQ confounding the associations, models were adjusted for age, sex, SES, and IQ.

Before answering the research question, children included and excluded from the analysis were compared in terms of association with age, sex, SES, and overall levels of psychopathology (CBCL total scores, SDQ difficulties and impact scores, and number of DSM-IV psychiatric diagnoses) and frequencies of SRPE, CRPE, and APS were described.

Comparative associations between nSRPE, nCRPE, and APS on the one hand and demographic characteristics (age, sex, and SES) and IQ on the other were tested. The four mutually exclusive groups (no PE, nSRPE, nCRPE, and APS) were recoded into dummies using “no PE” as the reference category. Subsequently, the comparative pattern of associations of nSRPE, nCRPE, and APS was probed by assessing the association of categories nSRPE, nCRPE, and APS with mental health (CBCL total scores, SDQ difficulties and impact scores, and number of DSM-IV psychiatric diagnoses) (5, 51), a measure of “positive behavioral attributes” (YSI) (45) and a measure of affective flattening, derived from the CAARMS, rated on a five-level scale of “absent,” “questionable,” “mild,” “moderate,” and “moderately severe.”

Post-estimation Wald tests were used to test differences between regression coefficients obtained from the same regression model.

Results

Sample Characteristics

Of the 2,511 children included in the study, 267 children did not attend psychological evaluation and four children who attended psychological evaluation had more than four items of the CAPE missing and were excluded. The 271 (10.8%) children who were excluded did not significantly differ from those who completed the evaluation in terms of demographic and clinical characteristics (age B = −0.16, 95% CI = −0.95 to 0.63, p = 0.69; sex OR = 0.9, 95% CI = 0.65–1.23, p = 0.5; SES B = 0.24, 95% CI = −1.61 to 2.09, p = 0.8; SDQ impact score B = −0.05, 95% CI = −0.41 to 0.31, p = 0.79; SDQ difficult scores = −0.33, 95% CI = −1.9 to 1.23, p = 0.68; CBCL total scores = −0.06, 95% CI = −0.14 to 0.02, p = 0.15; DSM-IV diagnosis B = 0.03, 95% CI = −0.07 to 0.13, p = 0.55; DAWBA positive attributes score B = 0.87, 95% CI = −1.56 to 3.3, p = 0.48).

Participants were aged 6–14 years (mean age = 10.4, SD = 1.9; Table 1) and 53% were male. The great majority of children, N = 2,010 (91%), were middle class or above. Overall levels of psychopathology were high and 27% of children had at least one DSM IV diagnosis. Mean IQ was 101.6 (SD = 16.7), which is similar to the normative mean, defined to be 100 for the general population. In general, parents rated their children’s positive attributes high (mean = 35.4, SD = 8.56). Psychologists considered that 108 children (5%) had mild to moderately severe blunted affect.

TABLE 1

Table 1 Sample characteristics.

Frequencies of SRPE, CRPE, and APS

The overall frequency of SRPE was high (Table 2). In total 1,040 (46%) children reported at least one PE that was reported at a frequency of at least “frequently” or at a distress level of at least “quite distressing.” The proportion of children with at least one PE was considerably lower if clinical judgment was used. Thus, 336 children (15%) were categorized as having at least one very likely or certain CRPE. There was a considerable overlap between categories (see Figure 1), and when the sample was divided into mutually exclusive categories of unusual experiences, 741 children (33%) had at least one SRPE but not CRPE or APS (here referred to as non-confirmed self-reported psychotic experiences, nSRPE), 237 children (11%) had at least one CRPE but not APS (here called non-APS clinically relevant Psychotic Experience, nCRPE), and 127 (6%) had APS (Table 3).

FIGURE 1

Figure 1 Frequencies of subjects across different categories of the presence of self-reported psychotic experiences (SRPE), clinically validated psychotic experiences (CRPE), and attenuated psychotic symptoms (APS).

TABLE 2

Table 2 Frequency of self-reported (SRPE) and clinically relevant psychotic experiences (CRPE) per item for the overall sample and subsamples according to recruitment methodology (increased familial levels of psychopathology and randomly selected).

TABLE 3

Table 3 Association of mutually exclusive categories of non-confirmed self-reported psychotic experiences (nSRPE), non-APS clinically relevant PE (nCRPE), and attenuated psychotic symptoms (APS) and demographic and clinical variables.

Differences Between nSRPE, nCRPE, and APS

Table 3 displays the regression coefficients and odds ratios for the associations between the three mutually exclusive categories of nSRPE, nCRPE, and APS on the one hand and clinical characteristics on the other. When the sample was stratified by mutually exclusive category of PE status, the nSRPE group had higher SDQ difficulties and CBCL total scores than the reference group (Table 3). Children in the nCRPE group had higher levels of psychopathology (SDQ difficulties, impact, CBCL, and DSM-IV diagnosis) and had significantly lower ratings of positive attributes than the reference category. APS, on the other hand, were significantly associated with all the hypothesized predictors; that is, they exhibited higher levels of psychopathology, less positive attributes, and higher levels of affective flattening.

Categories were generally directionally similarly associated in a dose–response fashion, with clinical attributes in the order of APS > nCRPE > nSRPE. nCRPE was significantly more strongly associated with SDQ impact scores and SDQ difficult scores than nSRPE (Table 3). APS were more strongly associated with SDQ impact and difficult scores, positive attributes, and affective flattening than nSRPE. APS were associated more strongly with less positive attributes and more severe blunted affect than nCRPE.

Discussion

The present study assessed the rates and comparative associations of SRPE, CRPE, and APS in a large non-clinical population of children at increased risk of mental disorders. The results showed that the use of clinical judgment instead of self-report resulted in an enhanced pattern of associations, but that SRPE rejected by clinicians are also associated with increased levels of general psychopathology. Also, we found that clinical validation of PE (CRPE) and higher levels of severity of PE (PE meeting APS criterion) impacted the pattern of associations with psychopathology, protective factors, and psychotic disorder-specific psychopathology. Thus, children and adolescents with CRPE had higher levels of psychopathology than those with SRPE. However, those with SRPE not validated by clinicians had higher SDQ impact scores and CBCL total scores than participants without SRPE. Another finding was that CRPE below and above the threshold for APS were not significantly different from each other in terms of associations with mental health problems in general. Although the use of the APS criterion applied to PE did not improve the prediction of psychopathology in general, its use led to the selection of a group of children with more severe affective flattening and less positive attributes. As APS, in fact, represents a severity subgroup of PE, the results support the idea of a spectrum phenotype including SRPE, CRPE, and APS, SRPE showing the weakest link to poor mental health and CRPE and APS showing stronger links with general psychopathology; however, only APS is associated with blunted affect and is more strongly linked with less positive attributes than the other two measurements.

Self-Report Versus Clinical Evaluation of PE

Usually, clinical evaluation is considered the gold standard for the assessment of psychiatric symptoms, and this is particularly true for psychotic patients and for children. In both populations, self-report is considered unreliable because of reduced capacity of judgment. Contradicting this idea, results from the present study showed that experiences considered by children as frequent and distressing, even when judged little likely to be a PE by psychologists, were associated with increased scores of general psychopathology. In practical terms, it means that those who report frequent and distressing experiences not validated by clinicians can be, although to a lesser degree, at increased risk of mental health problems.

When self-report is used to access PE, different items tend to covary (51–53) and subdomains are highly correlated (52, 54), suggesting that all reflect the same underlying construct. This latent construct has been shown to have validity in relation to clinical psychosis; self-reported PE is associated with measures of general psychopathology, environmental risk factors for psychosis, and family history for psychotic disorders (5, 55). Although the aforementioned data supports the reliability and validity of self-reported PE, very few studies assessed the validity of self-report against clinical judgment in children and adolescents. The two available studies that investigated validity and reliability of SRPE versus CRPE found surprisingly good psychometric properties for some items of self-report questionnaires (15, 16). In these studies, the positive predictive value of the self-report of hearing voices against interview-validated auditory hallucinations was around 70% (15, 16), and its sensitivity and specificity were respectively 67% and 92% (16). Evidence from studies in adult populations have shown that disregarding PE based on clinical impression can be misleading. Adults with SRPE not validated by psychological interview have more psychiatric morbidity, substance misuse, higher levels of neuroticism, and worse affective regulation and social functioning than individuals without PE (25). Additionally, SRPE considered non-relevant by clinicians was found to prospectively predict the development of clinical psychotic disorders among adults (14). Thus, clinical evaluation might not necessarily represent the gold standard for the assessment of PE. Results from the present study showed that children that report PE considered irrelevant by clinicians might be at increased risk of mental health problems. Although this association was small and the use of clinical evaluation did improve the sensitivity of PE for the identification of children with worse metal health, the use of clinical judgment led to the exclusion of children with increased levels of psychopathology.

Attenuated Psychotic Symptoms Versus Clinically Validated PE

Apart from the question on whether or not clinical evaluation is preferred over self-report, an additional issue on the evaluation of subclinical psychotic features is on how to set the threshold above which a given experience would be considered clinically relevant. In this study, two different strategies were tested, one broader (nCRPE) and another more exclusive (APS). From the clinical perspective, the identification of APS and CRPE follows different procedures. APS is based on setting a threshold to differentiate more severe forms of PE, and it takes into consideration not only subclinical delusions and hallucinations but also levels of speech organization. Although the use of a threshold to differentiate mild from more severe forms of PE, as is done for APS, increases the specificity of the criterion for the selection of those at higher risk of mental disorders, the threshold set by CAARMS (the most commonly used criterion for UHR) was arbitrarily defined (56). Although different authors have already proposed the use of broader phenotypes for the identification of high-risk individuals (57), to our knowledge, this study was the first to compare APS with milder forms of clinically validated PE in an epidemiological sample using clinical interviewing. We showed that, when compared to CRPE, the APS criterion failed to differentiate children with more or less severe psychopathology. Nevertheless, it was useful for the identification of a group of children with less positive attributes and more severe affective flattening. It is possible that the stronger association with affective flattening found for APS when compared to CRPE, which showed associations in the same direction but not statistically significant, is due to the fact that, differently from CRPE, APS considers information about severity or about speech organization, another characteristic of psychosis. This result is consistent with the model previously proposed, according to which PE represent transdiagnostic features and the combination of symptoms from different diagnostic constructs (i.e., positive symptoms, negative symptoms, and disorganization) allow for a more accurate classification of individuals into the categorical diagnoses of psychosis (58). Under this framework, the use of UHR criteria, combining not only PE but also other features of psychosis, can be considered an advantage in decreasing the pluripotency of the diagnosis towards a more specific phenotype to target prediction for psychosis (59).

Association of PE With Psychopathology and Positive Attributes

Previous studies showed an association between PE and general levels of psychopathology (13, 55, 60, 61). In accordance with this earlier work, we found that the validity of PE as an indicator of mental health problems is independent of the source of assessment of PE, after correction for IQ and demographic characteristics. The association with psychopathology in our sample does not seem to be specific, with both internalizing and externalizing mental health problems being associated with the three categories (see Supplementary Material for frequencies of DSMIV diagnosis by mutually exclusive categories of non-confirmed self-reported psychotic experiences (nSRPE), non-APS clinically relevant PE (nCRPE), and attenuated positive symptoms (APS) and for the association of CBCL subscales with these groups). We also showed that children and adolescents with increased rates of PE not only have increased levels of psychopathology but also display fewer positive attributes and that levels of positive attributes are important in the distinction between those with more and less severe subclinical psychotic features. This is of relevance for clinical practice because the psychiatric evaluation of children frequently emphasizes more the identification of symptoms and deficits rather than on the identification of abilities, even though positive attributes may act independently from psychopathology in promoting adaptation and functioning (62). Therefore, levels of positive attributes should be further studied as a moderator of risk of mental disorders.

Assessment of Self-Reported PE Among Young Children

There is evidence from previous research showing that, when tested against clinical judgment, young children are not reliable for the report of a range of psychiatric symptoms (44, 63). Although there is no available data on the validity of CAPE among young children, CAPE is frequently used to assess PE in youth, showing very good psychometric properties (64, 65). Indeed, CAPE seems to be more reliable among younger people. A meta-analysis of CAPE psychometric properties identified 111 studies and showed that the mean alpha values for studies with mean age lower than 25 years were more internally reliable than that of adults older than 25 years. In our sample, we found evidence that CAPE can be reliably assessed in younger children. First, the age of those with non-confirmed SRPE was not significantly lower than the age of those with clinically relevant PE (B = −0.08 (−0.33 to 0.17), p = 0.55; regression coefficients (B) and respective confidence intervals [CI], obtained from multilevel regression model testing non-confirmed SRPE against the reference category Clinically Relevant PE [N = 1,102] and adjusted for gender, socioeconomic status, and IQ). Secondly, the positive predictive value (PPV) of self-report against clinical judgment did not seem to increase with age (PVV was 27.5% for children bellow or equal 10 years old and 25.2% for older children—see Supplementary Material for more details about PPV of SRPE against clinical judgment and CRPE against APS). Although the significance of the difference between different values of PPV cannot be statistically tested, it was surprising to us that younger children had slightly better values of PPV. We do not believe, however, that younger children would be more reliable than the older ones in self-reporting PE; instead, it may reflect the better quality of interviews provided by older children. The evaluation of PE in pediatric population may be very challenging as children’s description of delusions are usually poorly elaborated and vague and may be built on real experiences (e.g., being teased or disliked), hallucinations are often multimodal, and children may give them names, which may be stereotypic (e.g., “the devil”) (66). Third, the internal consistency of CAPE self-report was surprisingly good when tested for the subsample of very young children (Cronbach’s alpha = 0.9 for children bellow 8 years, N = 259). It is possible that the fact that psychologists read CAPE questions to children increased the quality of children’s self-report, but there is also a possibility that young children can reliably provide information about their PE using CAPE self-ratings.

Limitations

This study has some limitations that should be addressed. First, the majority of children studied were selected based on having increased risk of mental health problems. Consequently, the associations with morbidity found here can be stronger than expected for the general population. Second, psychologists’ judgment relied exclusively on interview with children; parents were not interviewed by the psychologists. Consequently, only PE reported by children were evaluated clinically and potential false negatives were not assessed.

Third, it is also important to comment some methodological caveats involving the measurements used. The context and the methodology used to assess APS in the current epidemiological population-based setting are different from the one typically used in UHR studies with a focus on selected help-seeking individuals in the clinical setting. Different from most frequently used instruments for UHR, the precise frequency of symptoms was not considered because children and adolescents were not expected to reliably report the frequency of their experience with such a precision as required by the instruments for UHR. If the frequency of experiences had been taken into consideration, even more CRPE would have been rejected. Another important difference is the fact that the psychologists who performed the clinical evaluation received 4 h of training on CAARMS ratings; they were not experts in the evaluation of prodromal schizophrenia, as would be the case in an UHR clinical setting. Furthermore, it should be mentioned that the diagnosis of UHR considers not only APS but also other features like genetic risk and deterioration of global functioning; however, these are rare and the great majority of UHR, in fact, is based on APS (67). Furthermore, we used CAARMS ratings, but not the entire clinical interview, as it would not be feasible in such a large population-based study. Consequently, the methodology used for the assessment of APS was not as rigorous as that used in prodromal clinic studies. Another issue is that the CAPE Portuguese version was not validated previously, and although there is data showing that the CAPE has very good psychometric properties for adult populations (65), it has not yet been validated for the assessment of PE in children as young as some of the children in our sample (64). Furthermore, we had someone reading the questions to the children, which may limit the generalizability of the findings to samples where youth are administered written queries of PE. We also did not collect data showing the interrater reliability of psychologist assessment of CAPE and CAARMS. Besides these possible methodological caveats, the three instruments showed good psychometric properties in our sample. The internal consistency of CAPE self-report items was very good, with an alpha of 0.89, and the exclusion of none of the items would lead to better results. When items were dichotomized to allow classification of subjects into those with and without SRPE, the internal consistency decreased, but was still acceptable, alpha = 0.74, with the exclusion of none of the items leading to better alpha values. The internal consistency of the CAPE clinical scores was also very good, with an alpha of 0.83. When items were dichotomized, the alpha values decreased, but were still acceptable (alpha = 0.72), and the exclusion of none of the items would result in better alpha values. The internal consistency of the two items of CAARMS used to generate the APS criterion was also good (alpha = 0.81). It is also noteworthy that all the three measurements were found to be associated with hypothesized predictors, as shown by the present research, which reinforces the validity of the measurements used.

Finally, although PE were associated with measurements of poor mental health, the evidence presented here is insufficient to support the idea that all children with PE should be assessed clinically. For this aim, it would be important to estimate the negative and positive predictive validity of PE as an index of future and present mental health problems as well as to compare it with other indicators and to raise empirical data on the cost-efficacy, effectiveness, and ethics of interventions to improve mental health in those with such features.

Conclusions

In children and adolescents, self-reported PE not recognised by clinicians, clinically validated PE, and APS appear to tap into the same underlying construct, differing in predicted ways in degree and in kind as a function of clinical severity in terms of associations with psychopathology, positive attributes, indicators of the multidimensional psychotic syndrome, and demographics.

Ethics Statement

The studies involving human participants were reviewed and approved by Comissão para análise de projetos de pesquisa (CAPPesq) da diretoria clínica do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author Contributions

TM conceived and designed the study, conceived and designed the analysis, collected the data, performed the analysis, and wrote the paper. JvO and MD conceived and designed the analysis, performed the analysis, and wrote the paper. AG, PP, GS, GM, EM, LR, GP, and RB conceived and designed the study, collected the data, and provided critical feedback on the manuscript. JM and PM conceived and designed the study and provided critical feedback on the manuscript.

Funding

This work is supported by the National Institute of Developmental Psychiatry for Children and Adolescents, a science and technology institute funded by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq; National Council for Scientific and Technological Development; grant number 573974/2008-0) and Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP; Research Support Foundation of the State of São Paulo; grant number 2008/57896-8). It was also supported in part by the European Community’s Seventh Framework Program (grant agreement No. HEALTH-F2-2009-241909, Project EU-GEI). The author’s scholarships are supported by the following Brazilian government institutions: CNPq, FAPESP, Coordenação de Aperfeiçoamentode Pessoal de Nível Superior (CAPES; Brazilian Federal Agency for Support and Evaluation of postgraduate education) and Fundação de Amparo a Pesquisa do Estado do Rio Grande do Sul (FAPERGS; Research Support Foundation from the State of Rio Grande do Sul). Tais Moriyama received a sandwich scholarship from CAPES (process number 10553/12-6) and a PhD scholarship from CAPES; Ary Gadelha received a CAPES PhD scholarship; Pedro Mario Pan received a CNPq/CAPES master’s degree scholarship; Giovanni Abrahão Salum received a CAPES/FAPERGS post-doctoral scholarship; Gisele Gus Manfro, Jair J. Mari, EurípedesConstantino Miguel (302463/2011-9), Luis A. Rohde, Guilherme V. Polanczyk (310582/2017-2) and Rodrigo A. Bressan are in receipt of a senior research CNPq scholarship.

Conflict of Interest

AG has been a member of the speakers’ bureau/advisory board and/or acted as a consultant for Janssen-Cilag, Daiichi-Sankyo and Aché. PP received a CNPq PhD scholarship. LR has been a member of the speakers’ bureau/advisory board and/or acted as a consultant for Eli-Lilly, Janssen-Cilag. He also received travel awards from Novartis and Shire to attend the 2016 AACAP and the 2018 APA meetings. GP is employed by the University of São Paulo. He has received grant or research support from the Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP), the Fundação Maria Cecilia Souto Vidigal (FMCSV), Grand Challenges Canada, and the Bill and Melinda Gates Foundation. He has served as a consultant to Shire, Teva, and Medice. He has received royalties from Editora Manole. RB reports grants, personal fees, and non-financial support from Janssen and personal fees from Pfizer and from Ache outside the submitted work.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00782/full#supplementary-material

References

1. Jolley S, Kuipers E, Stewart C, Browning S, Bracegirdle K, Basit N, et al. The coping with unusual experiences for children study (CUES): a pilot randomized controlled evaluation of the acceptability and potential clinical utility of a cognitive behavioural intervention package for young people aged 8-14 years with unusual experiences and emotional symptoms. Br J Clin Psychol (2018) 57(3):328–350 doi: 10.1111/bjc.12176

PubMed Abstract | CrossRef Full Text | Google Scholar

2. National Institute for Health and Care Excellence. Psychosis and schizophrenia in children and young people: recognition and management. Manchester: National Institute for Health and Care Excellence. (2013). [Online] [Accessed on 17th October 2019] https://www.nice.org.uk/guidance/cg155.

Google Scholar

3. Miklowitz DJ, O’Brien MP, Schlosser DA, Addington J, Candan KA, Marshall C, et al. Family-focused treatment for adolescents and young adults at high risk for psychosis: results of a randomized trial. J Am Acad Child Adolesc Psychiatry (2014) 53(8):848–58. doi: 10.1016/j.jaac.2014.04.020

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Poulton R, Van Ryzin MJ, Harold GT, Chamberlain P, Fowler D, Cannon M, et al. Effects of multidimensional treatment foster care on psychotic symptoms in girls. J Am Acad Child Adolesc Psychiatry (2014) 53(12):1279–87. doi: 10.1016/j.jaac.2014.08.014

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kelleher I, Keeley H, Corcoran P, Lynch F, Fitzpatrick C, Devlin N, et al. Clinicopathological significance of psychotic experiences in non-psychotic young people: evidence from four population-based studies. British J Psychiatry (2012) 201(1):26–32. doi: 10.1192/bjp.bp.111.101543

CrossRef Full Text | Google Scholar

6. Clemmensen L, van Os J, Drukker M, Munkholm A, Rimvall MK, Vaever M, et al. Psychotic experiences and hyper-theory-of-mind in preadolescence–a birth cohort study. Psychol Med (2016) 46(1):87–101. doi: 10.1017/S0033291715001567

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Kaymaz N, Drukker M, Lieb R, Wittchen HU, Werbeloff N, Weiser M, et al. Do subthreshold psychotic experiences predict clinical outcomes in unselected non-help-seeking population-based samples? A systematic review and meta-analysis, enriched with new results. Psychol Med (2012) 42(11):2239–53. doi: 10.1017/S0033291711002911

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Kelleher I, Wigman JT, Harley M, O’Hanlon E, Coughlan H, Rawdon C, et al. Psychotic experiences in the population: association with functioning and mental distress. Schizophr Res (2015) 165(1):9–14. doi: 10.1016/j.schres.2015.03.020

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Poulton R, Caspi A, Moffitt TE, Cannon M, Murray R, Harrington H. Children’s self-reported psychotic symptoms and adult schizophreniform disorder: a 15-year longitudinal study. Arch Gen Psychiatry (2000) 57(11):1053–8. doi: 10.1001/archpsyc.57.11.1053

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Kelleher I, Corcoran P, Keeley H, Wigman JT, Devlin N, Ramsay H, et al. Psychotic symptoms and population risk for suicide attempt: a prospective cohort study. JAMA Psychiatry (2013) 70(9):940–8. doi: 10.1001/jamapsychiatry.2013.140

PubMed Abstract | CrossRef Full Text | Google Scholar

11. De Loore E, Gunther N, Drukker M, Feron F, Sabbe B, Deboutte D, et al. Persistence and outcome of auditory hallucinations in adolescence: a longitudinal general population study of 1800 individuals. Schizophr Res (2011) 127(1–3):252–6. doi: 10.1016/j.schres.2011.01.015

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bartels-Velthuis AA, de Willige G, Jenner JA, van Os J, Wiersma D. Course of auditory vocal hallucinations in childhood: 5-year follow-up study. British J Psychiatry (2011) 199(4):296–302. doi: 10.1192/bjp.bp.110.086918

CrossRef Full Text | Google Scholar

13. Scott J, Martin G, Bor W, Sawyer M, Clark J, McGrath J. The prevalence and correlates of hallucinations in Australian adolescents: results from a national survey. Schizophr Res (2009) 107(2–3):179–85. doi: 10.1016/j.schres.2008.11.002

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bak M, Delespaul P, Hanssen M, de Graaf R, Vollebergh W, van Os J. How false are “false” positive psychotic symptoms? Schizophr Res (2003) 62(1–2):187–9. doi: 10.1016/S0920-9964(02)00336-5

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Horwood J, Salvi G, Thomas K, Duffy L, Gunnell D, Hollis C, et al. IQ and non-clinical psychotic symptoms in 12-year-olds: results from the ALSPAC birth cohort. Br J Psychiatry (2008) 193(3):185–91. doi: 10.1192/bjp.bp.108.051904

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kelleher I, Harley M, Murtagh A, Cannon M. Are screening instruments valid for psychotic-like experiences? A validation study of screening questions for psychotic-like experiences using in-depth clinical interview. Schizophr Bull (2011) 37(2):362–9. doi: 10.1093/schbul/sbp057

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Schultze-Lutter F, Renner F, Paruch J, Julkowski D, Klosterkotter J, Ruhrmann S. Self-reported psychotic-like experiences are a poor estimate of clinician-rated attenuated and frank delusions and hallucinations. Psychopathol (2014) 47(3):194–201. doi: 10.1159/000355554

CrossRef Full Text | Google Scholar

18. Linscott RJ, van Os J. An updated and conservative systematic review and meta-analysis of epidemiological evidence on psychotic experiences in children and adults: on the pathway from proneness to persistence to dimensional expression across mental disorders. Psychol Med (2013) 43(6):1133–49. doi: 10.1017/S0033291712001626

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Yung AR, McGorry PD. The initial prodrome in psychosis: descriptive and qualitative aspects. Aust N Z J Psychiatry (1996) 30(5):587–99. doi: 10.3109/00048679609062654

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Kelleher I, Connor D, Clarke MC, Devlin N, Harley M, Cannon M. Prevalence of psychotic symptoms in childhood and adolescence: a systematic review and meta-analysis of opulation-based studies. Psychol Med (2012) 42(9):1857–63. doi: 10.1017/S0033291711002960

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Schultze-Lutter F, Michel C, Schmidt SJ, Schimmelmann BG, Maric NP, Salokangas RK, et al. EPA guidance on the early detection of clinical high risk states of psychoses. Eur Psychiatry (2015) 30(3):405–16. doi: 10.1016/j.eurpsy.2015.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Stefanis NC, Hanssen M, Smirnis NK, Avramopoulos DA, Evdokimidis IK, Stefanis CN, et al. Evidence that three dimensions of psychosis have a distribution in the general population. Psychol Med (2002) 32(2):347–58. doi: 10.1017/S0033291701005141

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ian K, Jenner JA, Cannon M. Psychotic symptoms in the general population - an evolutionary perspective. British J Psychiatry (2010) 197(3):167–9. doi: 10.1192/bjp.bp.109.076018

CrossRef Full Text | Google Scholar

24. Schmidt SJ, Schultze-Lutter F, Schimmelmann BG, Maric NP, Salokangas RK, Riecher-Rossler A, et al. EPA guidance on the early intervention in clinical high risk states of psychoses. Eur Psychiatry (2015) 30(3):388–404. doi: 10.1016/j.eurpsy.2015.01.013

PubMed Abstract | CrossRef Full Text | Google Scholar

25. van Nierop M, van Os J, Gunther N, Myin-Germeys I, de Graaf R, Have M, et al. Phenotypically continuous with clinical psychosis, discontinuous in need for care: evidence for an extended psychosis phenotype. Schizophr Bull (2012) 38(2):231–8. doi: 10.1093/schbul/sbr129

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Schimmelmann BG, Schultze-Lutter F. Early detection and intervention of psychosis in children and adolescents: urgent need for studies. Eur Child Adolesc Psychiatry (2012) 21(5):239–41. doi: 10.1007/s00787-012-0271-z

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Mossaheb N, Becker J, Schaefer MR, Klier CM, Schloegelhofer M, Papageorgiou K, et al. The Community Assessment of Psychic Experience (CAPE) questionnaire as a screening-instrument in the detection of individuals at ultra-high risk for psychosis. Schizophr Res (2012) 141(2–3):210–4. doi: 10.1016/j.schres.2012.08.008

PubMed Abstract | CrossRef Full Text | Google Scholar

28. McGorry PD, McFarlane C, Patton GC, Bell R, Hibbert ME, Jackson HJ, et al. The prevalence of prodromal features of schizophrenia in adolescence: a preliminary survey. Acta Psychiatry Scand (1995) 92(4):241–9. doi: 10.1111/j.1600-0447.1995.tb09577.x

CrossRef Full Text | Google Scholar

29. Barragan M, Laurens KR, Navarro JB, Obiols JE. ȁTheory of Mind’, psychotic-like experiences and psychometric schizotypy in adolescents from the general population. Psychiatry Res (2011) 186(2–3):225–31. doi: 10.1016/j.psychres.2010.07.051

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Yung AR, Phillips LJ, Yuen HP, Francey SM, McFarlane CA, Hallgren M, et al. Psychosis prediction: 12-month follow up of a high-risk (“prodromal”) group. Schizophr Res (2003) 60(1):21–32. doi: 10.1016/S0920-9964(02)00167-6

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Dominguez MD, Saka MC, Lieb R, Wittchen HU, Van Os J. Early expression of negative/disorganized symptoms predicting psychotic experiences and subsequent clinical psychosis: a 10-year study. Am J Psychiatry (2010) 167(9):1075–82. doi: 10.1176/appi.ajp.2010.09060883

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Kring AM, Moran EK. Emotional response deficits in schizophrenia: insights from affective science. Schizophr Bull (2008) 34(5):819–34. doi: 10.1093/schbul/sbn071

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Van Os J, Kapur S. Schizophrenia. Lancet (2009) 374(9690):635–45. doi: 10.1016/S0140-6736(09)60995-8

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Salum GA, Gadelha A, Pan PM, Moriyama TS, Graeff-Martins AS, Tamanaha AC, et al. High risk cohort study for psychiatric disorders in childhood: rationale, design, methods and preliminary results. Int J Methods Psychiatry Res (2015) 24(1):58–73. doi: 10.1002/mpr.1459

CrossRef Full Text | Google Scholar

35. Weissman MM, Wickramaratne P, Adams P, Wolk S, Verdeli H, Olfson M. Brief screening for family psychiatric history: the family history screen. Arch Gen Psychiatry (2000) 57(7):675–82. doi: 10.1001/archpsyc.57.7.675

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Konings M, Bak M, Hanssen M, van Os J, Krabbendam L. Validity and reliability of the CAPE: a self-report instrument for the measurement of psychotic experiences in the general population. Acta Psychiatry Scand (2006) 114(1):55–61. doi: 10.1111/j.1600-0447.2005.00741.x

CrossRef Full Text | Google Scholar

37. Yung AR, Nelson B, Baker K, Buckby JA, Baksheev G, Cosgrave EM. Psychotic-like experiences in a community sample of adolescents: implications for the continuum model of psychosis and prediction of schizophrenia. Aust N Z J Psychiatry (2009) 43(2):118–28. doi: 10.1080/00048670802607188

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Pesquisa ABdEd (2010) Critério de Classificação Econômica Brasil,. Associação Brasileira de Empresas de Pesquisa. http://www.abep.org/novo/Content.aspx?ContentID=301. Accessed 2013

Google Scholar

39. Paiva GFdS, N, and A., FC. Consumption and socioeconomic classification in Brazil: a study based on the Brazilian family expenditure survey. Paper presented at the IARIW-IBGE Conference on Income, Wealth and Well-Being in Latin America, 13 Sep 2013; Rio de Janeiro, Brazil: SDBd (2013).

Google Scholar

40. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry (2001) 40(11):1337–45. doi: 10.1097/00004583-200111000-00015

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Bordin IA, Rocha MM, Paula CS, Teixeira MC, Achenbach TM, Rescorla LA, et al. (2013) Child behavior checklist (CBCL), youth self-report (YSR) and teacher’s report form(TRF): an overview of the development of the original and Brazilian versions. Cad Saude Publica 29 (1):13–28 doi: 10.1590/S0102-311X2013000100004

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Goodman R, Ford T, Richards H, Gatward R, Meltzer H. The Development and well-being assessment: description and initial validation of an integrated assessment of child and adolescent psychopathology. J Child Psychol Psychiatry (2000) 41(5):645–55. doi: 10.1111/j.1469-7610.2000.tb02345.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Fleitlich-Bilyk B, Goodman R. Prevalence of child and adolescent psychiatric disorders in southeast Brazil. J Am Acad Child Adolesc Psychiatry (2004) 43(6):727–34. doi: 10.1097/01.chi.0000120021.14101.ca

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Schwab-Stone ME, Shaffer D, Dulcan MK, Jensen PS, Fisher P, Bird HR, et al. Criterion validity of the NIMH diagnostic interview schedule for children version 2.3 (DISC-2.3). J Am Acad Child Adolesc Psychiatry (1996) 35(7):878–88. doi: 10.1097/00004583-199607000-00013

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Vidal-Ribas P, Goodman R, Stringaris A. Positive attributes in children and reduced risk of future psychopathology. British J Psychiatry (2015) 206(1):17–25. doi: 10.1192/bjp.bp.114.144519

CrossRef Full Text | Google Scholar

46. Wechsler D (2002) WISC-III: Escala de Inteligência Wechsler para Crianças: Manual, vol 3a. ed. Casa do Psicólogo, São Paulo.

Google Scholar

47. Tellegen A, Briggs PF. Old wine in new skins: grouping Wechsler subtests into new scales. J Consulting Psychol (1967) 31(5):499–506. doi: 10.1037/h0024963

CrossRef Full Text | Google Scholar

48. Figueiredo VLM (2001) Uma adaptação brasileira do teste de inteligência WISC-III. Curso de Pós-Graduação em Psicologia, Instituto de Psicologia - Universidade de Brasília., Brasília, DF.

Google Scholar

49. StataCorp (2013) Stata Statistical Software., vol Release 13, StataSE edn. StataCorp LP., College Station, TX.

Google Scholar

50. Snijders T, Bosker R. Multilevel analysis, an introduction to basic and advanced modeling. London: SAGE Publications (1999).

Google Scholar

51. Wigman JT, Vollebergh WA, Raaijmakers QA, Iedema J, van Dorsselaer S, Ormel J, et al. The structure of the extended psychosis phenotype in early adolescence–a cross-sample replication. Schizophr Bull (2011) 37(4):850–60. doi: 10.1093/schbul/sbp154

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Laurens KR, Hobbs MJ, Sunderland M, Green MJ, Mould GL. Psychotic-like experiences in a community sample of 8000 children aged 9 to 11 years: an item response theory analysis. Psychol Med (2012) 42(7):1495–506. doi: 10.1017/S0033291711002108

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Barragan M, Laurens KR, Navarro JB, Obiols JE. Psychotic-like experiences and depressive symptoms in a community sample of adolescents. Eur Psychiatry (2011) 26(6):396–401. doi: 10.1016/j.eurpsy.2010.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Armando M, Nelson B, Yung AR, Ross M, Birchwood M, Girardi P, et al. Psychotic-like experiences and correlation with distress and depressive symptoms in a community sample of adolescents and young adults. Schizophr Res (2010) 119(1–3):258–65. doi: 10.1016/j.schres.2010.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Polanczyk G, Moffitt TE, Arseneault L, Cannon M, Ambler A, Keefe RS, et al. Etiological and clinical features of childhood psychotic symptoms: results from a birth cohort. Arch Gen Psychiatry (2010) 67(4):328–38. doi: 10.1001/archgenpsychiatry.2010.14

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Yung AR, Yuen HP, McGorry PD, Phillips LJ, Kelly D, Dell’Olio M, et al. Mapping the onset of psychosis: the comprehensive assessment of at-risk mental states. Aust N Z J Psychiatry (2005) 39(11–12):964–71. doi: 10.1111/j.1440-1614.2005.01714.x

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Simon AE, Umbricht D, Lang UE, Borgwardt S. Declining transition rates to psychosis: the role of diagnostic spectra and symptom overlaps in individuals with attenuated psychosis syndrome. Schizophr Res (2014) 159(2–3):292–8. doi: 10.1016/j.schres.2014.09.016

PubMed Abstract | CrossRef Full Text | Google Scholar

58. van Os J, Reininghaus U. Psychosis as a transdiagnostic and extended phenotype in the general population. World Psychiatry (2016) 15(2):118–24. doi: 10.1002/wps.20310

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Woods SW, Powers AR 3rd, Taylor JH, Davidson CA, Johannesen JK, Addington J, et al. Lack of diagnostic pluripotentiality in patients at clinical high risk for psychosis: specificity of comorbidity persistence and search for pluripotential subgroups. Schizophr Bull (2018) 44(2):254–63. doi: 10.1093/schbul/sbx138

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Laurens KR, Hodgins S, Maughan B, Murray RM, Rutter ML, Taylor EA. Community screening for psychotic-like experiences and other putative antecedents of schizophrenia in children aged 9-12 years. Schizophr Res (2007) 90(1–3):130–46. doi: 10.1016/j.schres.2006.11.006

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Cannon M, Caspi A, Moffitt TE, Harrington H, Taylor A, Murray RM, et al. Evidence for early-childhood, pan-developmental impairment specific to schizophreniform disorder: results from a longitudinal birth cohort. Arch Gen Psychiatry (2002) 59(5):449–56. doi: 10.1001/archpsyc.59.5.449

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Hoffmann MS, Leibenluft E, Stringaris A, Laporte PP, Pan PM, Gadelha A, et al. Positive attributes buffer the negative associations between low intelligence and high psychopathology with educational outcomes. J Am Acad Child Adolesc Psychiatry (2016) 55(1):47–53. doi: 10.1016/j.jaac.2015.10.013

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Fallon T Jr., Schwab-Stone M. Determinants of reliability in psychiatric surveys of children aged 6-12. J Child Psychol Psychiatry (1994) 35(8):1391–408. doi: 10.1111/j.1469-7610.1994.tb01282.x

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Lee KW, Chan KW, Chang WC, Lee EH, Hui CL, Chen EY. A systematic review on definitions and assessments of psychotic-like experiences. Early Interv Psychiatry (2016) 10(1):3–16. doi: 10.1111/eip.12228

CrossRef Full Text | Google Scholar

65. Mark W, Toulopoulou T. Psychometric properties of “community assessment of psychic experiences”: review and meta-analyses. Schizophr Bull (2016) 42(1):34–44. doi: 10.1093/schbul/sbv088

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Sikich L. Diagnosis and evaluation of hallucinations and other psychotic symptoms in children and adolescents. Child Adolesc Psychiatry Clin N Am (2013) 22(4):655–73. doi: 10.1016/j.chc.2013.06.005

CrossRef Full Text | Google Scholar

67. Fusar-Poli P, Van Os J. Lost in transition: setting the psychosis threshold in prodromal research. Acta Psychiatry Scand (2013) 127(3):248–52. doi: 10.1111/acps.12028

CrossRef Full Text | Google Scholar