Maren Reder
Ronja A. Runge
Renate Soellner- Institute of Psychology, University of Hildesheim, Hildesheim, Germany
Background: Communities That Care (CTC) is an evidence-based community change strategy for supporting healthy youth development. One of its key elements is the development of a community profile to identify and prioritise risk factors for health and behavioural problems in adolescents based on the CTC Youth Survey. The strategy was originally developed and evaluated in the United States. An adapted version of the survey has been used in Germany since its first implementation in 2008. However, the dimensionality and validity of the adapted risk factor scales have not yet been evaluated. Therefore, this study aimed to confirm the assumed unidimensional structure and to establish the concurrent criterion validity of each risk factor.
Methods: A sample of 1,911 adolescents attending grades six to eleven in Lower-Saxony, Germany, was used to evaluate 23 risk factor scales of the German CTC Youth Survey covering the domains peer/individual, family, school, and community. Confirmatory factor analysis was used to assess the dimensional structure of all risk factors with sufficient item numbers. Goodness of fit was determined using CFI, SRMR, and RMSEA. Latent regression analysis tested the concurrent criterion validity of all unidimensional risk factors. For this, violent and delinquent behaviour, substance use, and depressive symptomatology were regressed on each risk factor.
Results: All evaluable risk factors demonstrated acceptable to good model fit regarding unidimensionality and predicted violent and delinquent behaviour, and substance use better than depressive symptomatology. Regarding the peer/individual risk factors, there are particularly high correlations with violent and delinquent behaviour, and substance use. In contrast, two risk factors were not correlated with substance use, whilst two other risk factors were not correlated with depressive symptomatology.
Conclusion: Overall, the results indicate that most risk factors demonstrate unidimensionality and are valid in terms of concurrent criterion validity. Although some risk factors were not correlated with every outcome, they still predicted most outcomes, suggesting that the CTC Youth Survey is a viable tool for communities to assess their community risk profile. Risk factors that did not demonstrate unidimensionality or concurrent criterion validity should be monitored in future surveys and interpreted with caution until further evidence is available.
1 Introduction
Community-based prevention is an important strategy for supporting and facilitating healthy youth development. There are numerous preventive interventions available for narrowly defined outcomes, often with a clinical focus, rather than promoting healthy development in general. For instance, reviews on community-based prevention, aimed at adolescents, have been published on topics such as dental health (1), obesity (2), sexual violence (3), depression and anxiety (4), and self-harm and suicide (5). Whilst some preventive interventions focus on a single outcome, others yield multiple outcomes. Communities face the highly challenging task of deciding which of the various prevention interventions best fit their needs. Therefore, community prevention systems that assist communities in selecting the most appropriate prevention interventions, following a needs assessment, as well as coordinating prevention efforts, are vital. One community prevention system that is comprehensive (6), evidence-based (7), and cost-effective (8) is Communities That Care (CTC).
CTC was developed in the United States in the 1980s by the Social Development Research Group (9, 10). It assumes that several risk and protective factors influence the occurrence of potentially health-compromising problem behaviours. The risk and protective factors, taken into account in the CTC framework, were identified by reviewing studies on adolescent drug use, delinquency, and violence (11). The aims of CTC are to reduce risk factors and enhance protective factors in order to prevent adolescent problem behaviour (9, 12), and to create good conditions for a safe and healthy childhood (13) by supporting the communities to select appropriate evidence-based programmes.
The CTC Youth Survey is a crucial component of CTC. It is used to measure risk and protective factors and to create risk and protective factor profiles for communities, enabling them to select prevention interventions that are tailored to their specific needs. Beyond that, this survey also measures problem behaviours (14). The CTC Youth Survey is theoretically based on the Social Development Model, which is grounded in criminological theory (15). Within this model, hypotheses regarding problem behaviour are formulated on the basis of research results on risk and protective factors. Notably, the “problem behaviours” of the CTC Youth Survey cannot always be considered “behaviours,” (e.g., depressive symptomatology) (16). Therefore, we will refer to them as “problem areas” in the following.
The CTC Youth Survey was designed to be used by 6th to 11th-grade students in the United States. The original survey underwent rigorous development steps, including the formation of an item pool, cognitive pretesting, pilot testing in classrooms, selection of items and scales using data from a probability sample of children from public schools, and assessment of the reliability and validity of these risk and protective factors (11). The risk and protective factors that had been shown in previous research to be related to drug use, violence, and delinquency, in at least two longitudinal studies, covered constructs from four domains: peer/individual, family, school, and community. The hypothesised mechanism by which risk and protective factors affect outcomes is different. Protective factors do not only affect outcomes directly, but also may buffer detrimental effects on children (17). Since risk factors are more prominent in public health discussion, in this paper we will focus on the validity of all risk factors (whilst exploring protective and/or buffering effects of protective factors will be addressed in separate research).
The original U.S. survey instrument was revised to include 20 risk factors after qualitative and quantitative data reduction procedures. Further analysis confirmed the dimensionality and validity of all 20 risk factors in the original U.S. CTC Youth Survey (11). The dimensionality of the risk factors was assessed using a two-phase Principal Component Analysis (11). Validity was assessed by examining the relationships between each of the scales measuring risk and protective factors and problem behaviour outcomes. Five of the identified factors, namely Laws and Norms Favourable to Drug Use, Transitions and Mobility, Poor Family Management, Family Attachment, and Early Initiation of Antisocial Behaviour, did not meet the criterion of unidimensionality for some grade level by gender combinations. Overall, the authors concluded to retain each of these scales as a single scale. Correlations with problem areas turned out as hypothesised and highest with scales in the Peer-Individual domain (11). To date, to our knowledge, scales regarding problem behaviour were not tested regarding unidimensionality [except for depressive symptomatology which confirmed a single factor (18)].
In Germany, CTC was adapted, in 2008, as part of a pilot project (Social Prevention In Networks, SPIN) and first implemented and tested at three sites in Lower Saxony (19). The German version of the CTC Youth Survey was based on the 2006 U.S. version and the 2004 Dutch Survey (19). A pre-test was performed with two school classes in 2009 (19). An evaluation of the pilot project indicated that CTC was implementable in German communities (19). The questionnaire used in the CTC pilot project underwent further revisions and was subsequently administered in a state-wide representative survey in Lower Saxony in 2019.
To evaluate a scale, testing dimensionality and validity is essential (20). Different methods can be used to test this (21). For the CTC Youth Survey, the most appropriate method for assessing dimensionality is confirmatory factor analysis, and for assessing concurrent criterion validity, latent regression analysis. Unidimensionality is an important prerequisite for interpreting test results (22). If items are summed or aggregated into a mean score, each item should be a good indicator of the construct. Criterion validity of risk factors is crucial, particularly if decisions are based on survey results.
Although established for the U.S. survey, the dimensionality and validity of the scales in the German version of the CTC Youth Survey must be examined due to the cultural differences between the United States and Germany. For example, societal tolerance of underage drinking and early sexual activity differs between the United States and Europe (6), as do attitudes towards the use of substances such as tobacco or illegal drugs, and dropping out of school (6). Furthermore, it is possible that the translation of the U.S. items may have affected the validity of the included risk factors.
To date, only two studies, focusing on the family domain, have been published regarding the dimensionality or validity of the scales in the German version of the CTC Youth Survey. One of these studies evaluated the psychometric properties of the risk and protective factor scales in adolescents who were hospitalised following acute alcohol intoxication (23). Of the seven family domain factors, three demonstrated good psychometric properties as unidimensional scales. One factor, family attachment, showed a factor structure with two other factors: attachment to mother and attachment to father. The second study tested the family risk factor ‘family attachment’ with data from the representative CTC Youth Survey in Lower Saxony (24). The same two-factor solution, representing attachment to the mother and attachment to the father, emerged.
The CTC Youth Survey was adapted to the Dutch context in the Netherlands (25), and the first survey was conducted in 2001. Due to geographical proximity and cultural overlap, data from the Dutch version could be considered more comparable to the German version than other language versions. However, none of the publications on CTC in the Netherlands (25–27) cover validity. Thus, despite its regular use in Germany, most risk factors of the German version of the CTC Youth Survey have not yet been tested for their validity, leaving an important research gap.
To ensure the current and future use of the German version of the CTC Youth Survey, it is essential to evaluate dimensionality and validity of all risk factors within CTC practise (i.e., all risk factors equally used as potential predictors for all problem areas) in the general population for which the survey is intended. Therefore, this study aimed to achieve two objectives: (1) to confirm the assumed unidimensional structure of the risk factors and problem areas through confirmatory factor analysis and (2) to establish the concurrent criterion validity of each risk factor with four problem areas through latent regression analysis.
2 Materials and methods
2.1 Sampling and participants
To obtain a representative sample of 7,000 students in grades six to eleven in Lower Saxony, we used a stratified random process to select classes. The sample was designed to be representative across grades, school types, and population size of the community. Information on students, classes, and schools in the 2018/2019 school year was provided by the State Education Authority of Lower Saxony. Data on population size was obtained from the Federal Statistical Office of Germany. In 2019, in Lower Saxony, 426,168 students attended public and private schools in grades six to eleven, excluding vocational and special schools. The CTC Youth Survey invited a sample of 7,000 students selected through the above described stratified random process, of which 2,191 participated. Data of 280 students had to be excluded due to missing or implausible demographic information, large proportion of missing data, short time spent on the questionnaire and/or reporting dishonest responses. The final sample consisted of 1,911 students, of whom 45.3% were boys, 54.0% were girls, and 0.8% were other. The mean age was 14.2 years, and 93.4% were born in Germany, 0.3% in Turkey, 0.7% in Russia, 0.9% in Poland, and 4.6% in other countries. 19.8% of the students attended grade 6, 15.5% attended grade 7, 24.0% attended grade 8, 13.9% attended grade 9, 16.5% attended grade 10, and 10.3% attended grade 11. The mean self-rated socio-economic status was 6.4 (SD = 1.4) on a scale of 1 to 10, with larger values indicating higher socio-economic status.
2.2 Measures
Twenty-three risk factors covering the domains of peer/individual, family, school, and community and four problem areas from the 2019 CTC Youth Survey were analysed in this study (please refer to Supplementary File S1 for a list of items). Table 1 describes the risk factors for all domains. With 10 factors the individual/peer domain is the most important domain (rebelliousness, early initiation of antisocial behaviour, early initiation of drug use, attitudes favourable to drug use, attitudes favourable to antisocial behaviour, peer drug use, peer antisocial behaviour, peer rewards for antisocial behaviour, sensation seeking, perceived risks of drug use), six community domain risk factors (low neighbourhood attachment, community disorganisation, transitions and mobility, perceived availability of drugs, perceived availability of handguns, laws and norms favourable to antisocial behaviour), followed by five family domain risk factors (family history of antisocial behaviour, poor family management, family conflict, parental attitudes favourable to drug use, parental attitudes favourable to antisocial behaviour), and two school domain risk factors (academic failure, low commitment to school). Usual response options were NO!, no, yes, YES! and scaling provides no categorisation nor recoding unless indicated otherwise.
Table 1. Risk factor scales.
The problem areas include violence, delinquency, substance use, and depressive symptomatology. Violence was assessed with six items responding to the questions “Have you done the following things in the past 12 months?”: “intentionally broken something that does not belong to you?,” “joined a violent or criminal youth gang?,” “been involved in a fight?,” “attacked someone to seriously hurt him or her?,” “threatened someone to get money?,” “had a weapon with you (e.g., a knife)?.” Response options were yes/no.
Delinquency was assessed with seven items responding to the questions “Have you done the following things in the past 12 months?”: “stolen something from a store?,” “been arrested by the police?,” “sold stolen items?,” “stolen something at school?,” “sprayed graffiti on someone else’s property?,” “illegally downloaded music or movies from the internet?,” “bullied someone at school or online?.” Response options were yes/no.
Substance use was assessed with eight items responding to the questions “Have you ever tried such things yourself? If so, how often in the last 4 weeks?”: “beer,” “wine / sparkling wine,” “mixed drinks,” “hard liquor,” “cigarettes/tobacco (shisha, pipe, snus),” “hash/marijuana,” “other illegal drugs (ecstasy, speed, LSD, cocaine, crystal, or heroin),” “prescription drugs without a prescription from a doctor (e.g., tranquillisers, stimulants, or painkillers).” Response options were never, 0 times in the last 4 weeks, 1–2 times in the last 4 weeks, 3–5 times in the last 4 weeks, 6–9 times in the last 4 weeks, 10–19 times in the last 4 weeks, 20 times or more in the last 4 weeks. Additionally, the item “Please think back to the last 4 weeks. During this time, how often have you had 5 or more alcoholic drinks in one evening?” was used. Response options were: not at all, 1 to 2 times, 3 to 5 times, 6 to 9 times, 10 to 19 times, 20 times or more.
Depressive symptomatology was assessed with the items “Sometimes I think my life is worth nothing.”, “Sometimes I think I’m good for nothing.”, “I often think I’m a failure”, “For the past year, I’ve felt depressed or sad most days, although some days I’ve felt okay.” Response options were: NO!, no, yes, YES!.
2.3 Data collection
The 2019 CTC Youth Survey received approval from the State Education Authority of Lower Saxony and the Ethics Committee of the University of Hildesheim. Study information and parental consent letters were sent to the schools of the sampled classes. If the respective head teachers and class teachers agreed to participate, they would inform parents by handing out information and consent sheets about the study and ask for their written consent for their child’s participation. The signed declarations of consent from the parents were kept at the respective school. The CTC Youth Survey was conducted online and completed by students during a single school lesson. Non-participation did not have any negative consequences for the students. Students were permitted to stop the survey at any time and skip questions.
The 2019 CTC Youth Survey was programmed and hosted by the German Research Centre for Artificial Intelligence. Each teacher was provided with a list of 35 single use one-time passwords (OTPs) for the online questionnaire. Each OTP allowed for one response. The first page of the online questionnaire provided information about the study and asked for the students’ consent to participate (tick a box). On the second page, the students were given instructions on how to complete the questionnaire.
2.4 Statistical analysis
The ‘lavaan’ package version 0.6–11 (28) was used in R Studio version 1.1.463 for the analyses. The analysis consisted of two steps: Confirmatory Factor Analysis (CFA) and latent regression analysis. Residual correlations between the items were modelled based on a priori assumptions for all models. The analysis was performed separately for each risk factor since the CTC Youth survey was designed to assess separate risk factors, not risk factor domains.
Initially, we tested unidimensionality for scales with more than three items (risk factor and problem area scales). Four independent raters (RR, HS, MR, NF) evaluated a priori which residual correlations were to be expected based on similarity of item content (e.g., “drunk beer” and “drunk wine/champagne”). Residual correlations that were expected by at least two of the four raters were included and estimated in the CFA models (i.e., unequal from zero). The fit of a one-factor solution was evaluated for each scale.
We used the Weighted Least Square Mean and Variance Adjusted Estimator (WLSMV) to estimate model parameters, as recommended for ordinal data with medium sample sizes (29). Scale setting was achieved using the fixed factor method. The model fit was evaluated using the likelihood ratio test statistic (χ2), the Comparative Fit Index (CFI), the Root Mean Square Error of Approximation (RMSEA), and the Standardised Root Mean Residual (SRMR). To interpret our results, we refer to the most conservative values: CFI: > 0.95 (30, 31); RMSEA: < 0.06 (30) with CI lower bound ≤ 0.05 and upper bound ≤ 0.10 though according to the authors model specifications, degrees of freedom, and sample size influence the usefulness of a set cut-off (32); SRMR: < 0.08 (30). Model fit was interpreted as overall acceptable, if 2 out of these 3 indicated acceptable model fit. Pairwise deletion was used for missing data. The robust model fit indices of ‘lavaan’ are reported.
In the second step, we conducted latent regression analysis to model a direct path in SEM between the latent risk factor and the latent outcomes. If the CFAs yielded satisfactory model fits, the respective risk factors were included in the analyses of concurrent criterion validity. Based on CTC practise, for each risk factor, we modelled relationships with all problem areas (i.e., delinquency, violence, depressive symptomatology, and substance use) in one model. To evaluate the concurrent criterion validity of the CTC risk factors, we used model fit as well as the significance and size of standardised β of regression paths.
At this point, it is important to note that scales with less than four items, such as IR1, IR10, FR3, FR4, SR1, CR1, CR5, and CR6, could not be tested for unidimensionality. However, we conducted an exploratory test of their concurrent criterion validity as described above. In order to calculate the concurrent criterion validity of the one-item risk factor perceived availability of handguns (CR5), we dichotomised the item into low (very difficult, difficult) and high (easy, very easy).
3 Results
3.1 Dimensionality
The CFI shows that all 15 evaluable risk factors and four problem areas achieved good model fit (please refer to Table 2 for the model fit and Supplementary File S1 for item wording and λ). However, for six scales, one of the three model fit indices indicated unacceptable fit. Specifically, attitudes favourable to drug use (IR5), attitudes favourable to antisocial behaviour (IR6), low commitment to school (SR2), and community disorganisation (CR2) showed unacceptable model fit with an RMSEA of 0.8–0.9. Additionally, the model fit for early initiation of antisocial behaviour (IR3) was deemed unacceptable based on the SRMR. Similarly, early initiation of drug use (IR4) only achieved good model fit after excluding two items (f37.8.2, other illegal drugs; f37.10.2, prescription drugs without a prescription). However, all other nine scales demonstrated good model fit according to all model fit indices. Regarding local model fit, all items had standardised loadings (λ) greater than 0.4, indicating adequate (33) item discrimination (see Supplementary File S1).
Table 2. Unidimensionality of risk factors and problem areas.
3.2 Validity
Table 3 presents the concurrent criterion validity of all unidimensional risk factors. Table 4 displays the exploratory analysis of concurrent criterion validity of risk factors with three or fewer items, for which dimensionality could not be assessed. Only moderate to strong correlations (β > 0.3) are bolded. All model fits for the concurrent criterion validity calculations are contained in Supplementary File S2.
Table 3. Concurrent criterion validity of unidimensional risk factors.
Table 4. Concurrent criterion validity of risk factors with 3 or less items.
For certain combinations of risk factors and problem areas, the model did not converge due to their correlation being at the border of the parameter space (i.e., approaching 1). This occurred with early initiation of antisocial behaviour (IR3) in relation to violence and delinquency, as well as with early initiation of drug use (IR4) in relation to substance use.
All peer/individual risk factors for which calculation was possible showed moderate to strong associations with violence, delinquency, and substance use. For depressive symptomatology, this was only the case for rebelliousness (IR1). Perceived risks of drug use (IR12, β = 0.050, p = 0.069) was the only individual risk factor that did not show a significant correlation with depressive symptomatology.
Out of the five family risk factors, family history of antisocial behaviour (FR1), poor family management (FR2), and parental attitudes favourable to antisocial behaviour (FR5) demonstrated moderate to strong correlations with violence and substance use. Additionally, substance use was correlated with parental attitudes favourable to drug use (FR4). All five family risk factors showed moderate to strong correlations with delinquency, but only family conflict (FR3) was correlated with depressive symptomatology.
Amongst the two school risk factors, academic failure (SR1) was found to be correlated with delinquency and depressive symptomatology, whilst low commitment to school (SR2) showed moderate correlations with violence, delinquency, and substance use.
Out of the six community risk factors, community disorganisation (CR2), perceived availability of drugs (CR4), and perceived availability of handguns (CR5) showed moderate to strong correlations with violence, delinquency, and substance use. Depressive symptomatology was only moderately correlated with one risk factor (low neighbourhood attachment, CR1). Perceived availability of handguns (CR5) did not show a significant correlation (β = 0.069, p = 0.326) with depressive symptomatology. There were no significant associations of at least moderate level between transitions and mobility (CR3) and laws and norms favourable to antisocial behaviour (CR6). Furthermore, these two risk factors (CR3, β = 0.059, p = 0.193; CR6, β = 0.020, p = 0.584) did not exhibit a significant correlation with substance use.
In summary, violence, delinquency and substance use are more predictable through the risk factors than depressive symptomatology; correlations between the risk factors and depressive symptomatology are weaker. The risk factors for family, school, and community vary between low and moderate to high correlations, whilst for the peer/individual risk factors, all correlations with violence, delinquency, and substance use are moderate to high.
4 Discussion
4.1 Dimensionality
In this initial evaluation of the hypothesised unidimensionality of all risk factors in the German CTC Youth Survey, most scales fit well according to all assessed model fit indices. For some scales, only two out of three model fit indices indicated sufficient model fit. Early initiation of antisocial behaviour (IR3) has very low n on most items, meaning that all ordinal categories except ‘no onset’ have very few cases. Therefore, it is questionable whether these items should be dichotomised (no onset / onset) or not assessed at all due to the low n for identifying (sub-)populations at risk.
Attitudes favourable to drug use (IR5) has been modified since the 2019 CTC Youth Survey removing the Item “favourable attitude towards using prescription drugs without prescription” as this substance was of low prevalence in the surveyed age groups and not part of the original CTC Youth Survey. This may improve IR5 model fit in analyses of subsequent survey data. Attitudes favourable to antisocial behaviour (IR6) may be problematic for building a mean scale score as it combines favourable attitudes towards skipping school (Item f19.9 “skips school” shows the lowest loading of this risk factor) with favourable attitudes towards violence and delinquency.
After the 2019 CTC Youth Survey, low commitment to school (SR2) has been modified by removing the item “number of days school was skipped.” This change was made because this item overlaps with the problem behaviour of “missing school,” and using it to build the risk factor and problem area simultaneously is not recommended. This modification may improve the SR2 model fit in analyses of subsequent survey data.
Community disorganisation (CR2) might be problematic as it measures specific problems in the neighbourhood (e.g., graffiti, fights) but also contains one item on general feeling of safety (“I feel safe in my neighbourhood”). This item shows the lowest loading. Excluding or isolating this item as a separate risk factor might be advisable for future surveys.
Additionally, modifications were made to one scale (IR4). Specifically, items f37.8.2 (other illegal drugs) and f37.10.2 (prescription drugs without a prescription) were excluded from the scale due to the low number of students reporting early onset of these substances. It is important to note that this scale was not included in either the original CTC Survey (11) or the current U.S. Survey (16).
Overall, we accepted all factors as hypothesised to be unidimensional because the pattern of model fit (i.e., at least 2 out of 3 fit indices) and local fit was overall favourable. When comparing our results to previous research on the CTC Youth Survey, we found a heterogeneous array of studies. A systematic review (34) examined the reliability and validity of the CTC Youth Survey and reported the following: Nine studies have been conducted in the United States (six of those for adaptation for other populations / cultural contexts) (11, 14, 35–41), three studies in Germany (23, 24, 42) and one each in South Africa (43), Colombia (44), Iran (45), Malaysia (46), and Trinidad and Tobago (47). These studies differ not only by country but also by the assessed domains of risk and protective factors, type of factor analysis used (34), and the composition and number of risk and protective factors in the assessed domains. Thurow et al. (34) conclude in their review that construct validity was generally adequate (although that implies that there were scales for which construct validity was not given) over diverse populations.
In previous research, some risk factors have similarly shown to be problematic regarding unidimensionality. Arthur et al. (11) calculated separate CFAs for each grade-sex combination for the 20 scales. The majority of the risk factors yielded satisfying results with regard to unidimensionality (11). However, early initiation of antisocial behaviour (IR3) also showed problems in its unidimensionality in that study (11). Accordingly, IR3 appears to be somewhat problematic regarding unidimensionality in both the U.S. version (11) and the German version. In both versions, this evidence was not as consistent (not all model fit indices, not all subgroups) as to warrant changing or removing this risk factor.
4.2 Validity
To distinguish the results, we considered associations to be present only if they showed at least a moderate strength of association. Nine out of 10 peer/individual risk factors were associated with violence, delinquency, and substance use indicating that peer/individual risk factors are good predictors of problem behaviour. In contrast, only one of the 10 peer/individual risk factors (rebelliousness) was associated with depressive symptomatology. Peer and individual risk factors have limited predictability on depressive symptomatology compared to problem behaviour. Most risk factors have a clear face validity linked to substance use or antisocial behaviour (directly assessing something related to these outcomes, e.g., peer drug use). However, rebelliousness (IR1) and sensation seeking (IR10) do not have a clear link to these outcomes.
Depending on the outcome, three to all of the five family risk factors predicted violence, substance use and delinquency. Depressive symptomatology was only predicted by family conflict (FR3). Again, most family risk factors are content-wise aiming at either substance use or antisocial behaviour. Only family conflict (FR3) and poor family management (FR2) have a different focus. Arthur et al. (11) assessed only correlations with the problem behaviours substance use and delinquency and report that poor family supervision, poor family discipline, family history of antisocial behaviour, and family attitudes favourable to antisocial behaviour were all moderately correlated with these.
Only delinquency was predicted by both school risk factors (academic failure and low school commitment); all other outcomes were predicted by only one school risk factor. Neither of these factors has a content-wise link to substance use or antisocial behaviour. In contrast, both school risk factors were moderately correlated with problem behaviours in previous research (11).
Community risk factors are good predictors of violence, delinquency, and substance use (three out of six; community disorganisation, CR2, perceived availability of drugs, CR4, perceived availability of handguns, CR5). Similarly, in previous research “perceived availability of drugs” showed one of the strongest associations with substance use and delinquency, whilst contrastingly, “community disorganisation” failed to show moderate correlations (11). Depressive symptomatology was only predicted by low neighbourhood attachment (CR1). Interestingly, a third pattern emerges in this domain: we find two risk factors not at least moderately predicting any outcome (transitions and mobility, CR3, and laws and norms favourable to antisocial behaviour, CR6). These were the only two predictors that were not at least moderately associated to one of the four outcomes. Thus, the usefulness of “transitions and mobility” and “laws and norms favourable to antisocial behaviour” as prediction factors is questionable.
No significant prediction could be made based on the following risk factors. There was no correlation between transitions and mobility (CR3) or laws and norms favourable to antisocial behaviour (CR6) and substance use. Additionally, there was no correlation between perceived risks of drug use (IR12) or perceived availability of handguns (CR5) and depressive symptomatology. Community risk factors seem to be less consistent predictors of problem behaviour and depressive symptomatology; possibly because the community context is furthest from the individual. Perceived risks of drug use (IR12), from a face validity point of view, seems to be a reasonable predictor of substance use but not depressive symptomatology. Previous research has shown that all risk factors have significant positive correlations with substance use and delinquency (11). Correlations were moderate to strong, with the exception of “community disorganisation” and “transitions and mobility” (11). “Transitions and mobility” therefore, appear to be problematic in both the German and the U.S. version.
The model could not be calculated for early initiation of antisocial behaviour (IR3) with violence and delinquency, as well as for early initiation of drug use (IR4) with substance use, due to the correlation approaching 1. A nearly perfect, though not very informative, correlation can be expected for current behaviour and early onset of the same behaviour.
In summary, the previous discussion points lead to the following overall conclusions: firstly, depressive symptomatology is not predicted as well as other problem areas. During the development phase of the CTC Youth Survey (11), risk factors were identified through literature reviews on adolescent drug use, delinquency, and violence, but not depressive symptomatology, which may explain why these are better predicted. Based on CTC practise, this has been overruled by communities using risk factor profiles to achieve better problem area outcomes (i.e., specific predictions of certain risk factors on only some problem areas are not taken into account). New problem areas, such as victim/survivor experience and indicators of wellbeing, have been identified over the years. These areas do not fit into the category of problem behaviour, and therefore lack strong theoretical underpinnings for the identification of relevant risk factors. The CTC Youth Survey initially included six outcomes: violence, delinquency, school drop-out, substance use, teenage pregnancy, and depressive symptomatology (11). However, previous CTC studies typically only report results on substance use and antisocial behaviour, rather than other CTC outcomes such as depressive symptomatology.
For instance, the European Monitoring Centre for Drugs and Drug Addiction (6) conducted a review on the reduction of substance use (incidence and prevalence), delinquency, and other problem behaviours. The review concluded that publications from the Community Youth Development Study, a randomised controlled community trial with 24 matched communities in the United States, mostly focused on substance use and delinquency/violence as outcomes (48). Similarly, the Pennsylvania Youth Survey (38) compared only substance use and delinquency outcomes between CTC and non-CTC communities.
The Alcohol Action in Rural Communities project (49), a cluster randomised controlled trial comprising 20 communities in Australia, focused solely on alcohol-related outcomes such as crime, traffic incidents, hospital inpatient admissions, risky consumption, and verbal abuse. Therefore, we recommend studying CTC outcomes such as depressive symptomatology, which have been part of the CTC Youth Survey from the beginning. Additionally, we suggest developing and including risk factors that predict these outcomes. Consideration should also be given to adding more outcomes related to wellbeing. We propose renaming the categorisation of outcomes, as becoming a victim or experiencing depressive symptoms should not be grouped as ‘problem behaviours’.
The second main conclusion is that correlations for problem behaviours are consistently higher for peer/individual risk factors than for other domains. This suggests that conceptualising the domains as a sequence ranging from proximal to distal may be valuable. As all outcomes are asking for individual behaviour/wellbeing/experience, a more direct (and thus stronger) criterion validity of peer/individual risk factors that ask for risks pertaining to the individual and not the community, family or school setting (etiologically, developmentally, temporally) is expectable (11). A previous study, in Germany, examined the transferability of risk factor cut-points (mean absolute deviation of the median) for substance use in terms of sensitivity, specificity, and criterion validity (42). Although our analyses used latent constructs instead of cut-points, it is worth noting that criterion validity was reported to be high, particularly for the peer/individual risk factors (42). Risk factors in the peer/individual domain showed higher correlations with problem behaviours than in the other domains (11). Additionally, community risk factors are not consistently predictive of problem behaviour and depressive symptomatology, possibly due to their distance from individual problem areas.
Early onset of a behaviour should not be modelled as a risk factor because past and present behaviours are nearly identical and previous behaviour is always a strong predictor of current behaviour, as shown e.g., in studies on the reasoned action approach. However, these scales showed concurrent criterion validity as risk factors for all other outcomes. It is debatable whether the onset of a problem behaviour should be interpreted as a risk factor. The onset of a behaviour is a necessary but not sufficient condition for developing problem behaviour. Therefore, we suggest removing these as risk factors from the CTC Youth Survey.
Finally, we found that every risk factor was correlated significantly with at least three problem areas. Therefore, one may conclude that concurrent criterion validity has been established for the German CTC Survey. However, when considering only moderate-to-strong relations, transitions and mobility (CR3) and laws and norms favourable to antisocial behaviour (CR6) did not sufficiently predict any outcome. Therefore, it may be considered to remove these risk factors from the German CTC survey.
4.3 Limitations
The CTC Youth Survey has undergone continuous adaptation in both its U.S. and German versions, including changes to both the scale and item levels. Some scales of the survey have been modified over time, with problematic factors being excluded and new ones being added (34). Therefore, when comparing the results of the original version by Arthur et al. (11) with the 2019 German version, some scales may differ. The current U.S. version (16) differs even more from the German version than the original final CTC Survey (11). Adaptations have also been made at the item level, which means that when comparing validity across studies, some items may diverge for reasons other than translation alone.
The data in this study were cross-sectional, therefore no causal relations between risk factors and problem areas can be assumed. It is worth noting that the majority of studies assessing the validity of the CTC Youth Survey have also been cross-sectional (17 out of 20; 34).
This study focuses solely on risk factors, as they may have a different prediction process than protective factors (“buffering”; 50). Additionally, we only assess the validity of single risk factors. Research on environmental risk factors in infancy suggests that the number of risk factors may be a better predictor of problem behaviour than singular risk factors (51).
The nested structure of the data (students within schools) was not considered, because the information on the schools was not available. According to the results of the multilevel logistic regression models in other research, some risk and protective factors showed significant variation across schools (11). It seems that there are significant risk/protection components at both levels (11).
Confounding variables cannot be excluded. The study only examined risk factors as single predictors and did not include additional sociodemographic variables. According to the CTC framework, these risk factors are universal (e.g., across grades).
The study is based on self-reported data. Whilst some items inquire about sensitive information and stigmatised behaviours, self-reported data may be more appropriate than parent-reported data when it comes to substance use.
4.4 Practical implications
The CTC Youth Survey aims to evaluate prevention needs (11), which is crucial for strategic prevention planning (11). It is important to ensure that the survey meets the validity requirements in the cultural context in which it is conducted. Communities are under increasing pressure to determine which prevention programmes should be implemented locally (6). To ensure coordinated and comprehensive prevention efforts, an effective needs assessment is required.
4.5 Future research
Future research should address the following topics: assessing the predictive criterion validity, as cross-sectional data only allows for the examination of correlative relationships and thus concurrent criterion validity. Additionally, the multilevel structure of the data should be taken into account. Furthermore, the validity of protective factors should be assessed, and their buffering effect should be modelled.
It remains unclear how CTC compares to other community-based interventions, or even which interventions are comparable. Gavine et al. (52) published a Cochrane protocol for “Universal community-based social development interventions for preventing community violence by young people,” but the editorial group later withdrew it. Therefore, we still lack a systematic review of this type of intervention.
Data availability statement
The datasets presented in this article are not readily available because data are available upon request due to ethical restrictions. Interested researchers may submit requests to the corresponding author for access to the data. All requests will be assessed by the Data Protection Officer of the University. Requests to access the datasets should be directed to Maren Reder, rederm@uni-hildesheim.de.
Ethics statement
The study involving humans was approved by the Ethical Committee of the University of Hildesheim. The study was conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants' legal guardians/next of kin.
Author contributions
MR: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing. RR: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing. HS: Conceptualization, Data curation, Formal analysis, Methodology, Validation, Writing – original draft, Writing – review & editing. RS: Conceptualization, Funding acquisition, Methodology, Supervision, Validation, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. Data collection was funded by the Crime Prevention Council of Lower Saxony. The funding body did not have any role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Acknowledgments
We acknowledge support of the publication fee by the Open Access Publication Funds of the University of Hildesheim. Nele Feierabend rated the residual correlations between the items and helped in the literature search for this manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2024.1472347/full#supplementary-material
References
1. Tonmukayakul, U, Sia, K-L, Gold, L, Hegde, S, Silva, AM, and Moodie, M. Economic models of preventive dentistry for Australian children and adolescents: a systematic review. Oral Health Prev Dent. (2015) 13:481–94. doi: 10.3290/j.ohpd.a35005
2. Gracia-Marco, L, Vicente-Rodríguez, G, Borys, JM, Le Bodo, Y, Pettigrew, S, and Moreno, LA. Contribution of social marketing strategies to community-based obesity prevention programmes in children. Int J Obes. (2011) 35:472–9. doi: 10.1038/ijo.2010.221
3. Lundgren, R, and Amin, A. Addressing intimate partner violence and sexual violence among adolescents: emerging evidence of effectiveness. J Adolesc Health. (2015) 56:S42–50. doi: 10.1016/j.jadohealth.2014.08.012
4. Christensen, H, Pallister, E, Smale, S, Hickie, IB, and Calear, AL. Community based prevention programs for anxiety and depression in youth: a systematic review. J Prim Prev. (2010) 31:139–70. doi: 10.1007/s10935-010-0214-8
5. Morken, IS, Dahlgren, A, Lunde, I, and Toven, S. The effects of interventions preventing self-harm and suicide in children and adolescents: an overview of systematic reviews. F1000Res. (2019) 8:890. doi: 10.12688/f1000research.19506.2
6. European Monitoring Centre for Drugs and Drug Addiction . Communities that care (CTC): a comprehensive prevention approach for communities. Publications office. Lisbon: European Monitoring Centre for Drugs and Drug Addiction (2017).
7. Chilenski, SM, Frank, J, Summers, N, and Lew, D. Public health benefits 16 years after a statewide policy change: communities that Care in Pennsylvania. Prev Sci. (2019) 20:947–58. doi: 10.1007/s11121-019-01028-y
8. Kuklinski, MR, Briney, JS, Hawkins, JD, and Catalano, RF. Cost-benefit analysis of communities that care outcomes at eighth grade. Prev Sci. (2012) 13:150–61. doi: 10.1007/s11121-011-0259-9
9. Hawkins, JD, Catalano, RF, and Miller, JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychol Bull. (1992) 112:64–105. doi: 10.1037/0033-2909.112.1.64
10. Röding, D, Soellner, R, Reder, M, Birgel, V, Kleiner, C, Stolz, M, et al. Study protocol: a non-randomised community trial to evaluate the effectiveness of the communities that care prevention system in Germany. BMC Public Health. (2021) 21:1927. doi: 10.1186/s12889-021-11935-x
11. Arthur, MW, Hawkins, JD, Pollard, JA, Catalano, RF, and Baglioni, A. Measuring risk and protective factors for use, delinquency, and other adolescent problem behaviors: the communities that care youth survey. Eval Rev. (2002) 26:575–601. doi: 10.1177/0193841X0202600601
12. Hawkins, JD, Catalano, RF, Kosterman, R, Abbott, R, and Hill, KG. Preventing adolescent health-risk behaviors by strengthening protection during childhood. Arch Pediatr Adolesc Med. (1999) 153:226–34. doi: 10.1001/archpedi.153.3.226
13. Kim, BKE, Gloppen, KM, Rhew, IC, Oesterle, S, and Hawkins, JD. Effects of the communities that care prevention system on youth reports of protective factors. Prev Sci. (2015) 16:652–62. doi: 10.1007/s11121-014-0524-9
14. Arthur, MW, Briney, JS, Hawkins, JD, Abbott, RD, Brooke-Weiss, BL, and Catalano, RF. Measuring risk and protection in communities using the communities that care youth survey. Eval Program Plann. (2007) 30:197–211. doi: 10.1016/j.evalprogplan.2007.01.009
15. Catalano, RF, Kosterman, R, Hawkins, JD, Newcomb, MD, and Abbott, RD. Modeling the etiology of adolescent substance use: a test of the social development model. J Drug Issues. (1996) 26:429–55. doi: 10.1177/002204269602600207
16. The Center for Communities That Care . 2014 youth survey scale dictionary. Seattle, WA: University of Washington (2014).
17. Lösel, F, and Farrington, DP. Direct protective and buffering protective factors in the development of youth violence. Am J Prev Med. (2012) 43:S8–S23. doi: 10.1016/j.amepre.2012.04.029
18. Rhew, IC, Monahan, KC, Oesterle, S, and Hawkins, JD. The communities that care brief depression scale: psychometric properties and criterion validity. J Community Psychol. (2016) 44:391–8. doi: 10.1002/jcop.21766
19. Schubert, H . Evaluation des Modellproprogramms “Communities That Care” in Niedersachsen: Theoretische Grundlagen und Empirische Befunde zur Sozialräumlichen Prävention in Netzwerken. Cologne: Verlag Sozial-Raum-Management (2013).
20. Boateng, GO, Neilands, TB, Frongillo, EA, Melgar-Quiñonez, HR, and Young, SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. (2018) 6:149. doi: 10.3389/fpubh.2018.00149
21. Wirtz, M . Konstruktvalidität, Prüfverfahren In: M Wirtz , editor. Dorsch – Lexikon der Psychologie. Göttingen: Hogrefe Verlag (2021)
22. Ziegler, M, and Hagemann, D. Testing the unidimensionality of items. Eur J Psychol Assess. (2015) 31:231–7. doi: 10.1027/1015-5759/a000309
23. Kuttler, H, Schwendemann, H, and Bitzer, EM. Familial risk and protective factors in alcohol intoxicated adolescents: psychometric evaluation of the family domain of the communities that care youth survey (CTC) and a new short version of the childhood trauma questionnaire (CTQ). BMC Pediatr. (2015) 15:191–14. doi: 10.1186/s12887-015-0471-z
24. Schwendemann, HE, Frisch, JU, Reder, M, Mößle, T, Soellner, R, and Bitzer, EM. The communities that care (CTC) family attachment scale: measurement invariance across family structures. J Child Fam Stud. (2018) 27:1440–8. doi: 10.1007/s10826-017-1009-9
25. Jonkman, HB, Junger-Tas, J, and Dijk, B. From behind dikes and dunes: communities that Care in the Netherlands. Child Soc. (2005) 19:105–16. doi: 10.1002/chi.865
26. Jonkman, HB, Haggerty, KP, Steketee, M, Fagan, A, Hanson, K, and Hawkins, JD. Communities that care, core elements and context: research of implementation in two countries. Soc Dev Issues. (2009) 30:42–57.
27. Jonkman, H, Aussems, C, Steketee, M, Boutellier, H, and Cuijpers, P. Prevention of problem behaviours among adolescents: the impact of the communities that care strategy in the Netherlands (2008–2011). Int J Dev Sci. (2015) 9:37–52. doi: 10.3233/DEV-13121
28. Rosseel, Y . Lavaan: an R package for structural equation modeling. J Stat Softw. (2012) 48:1–36. doi: 10.18637/jss.v048.i02
29. Brown, TA . Confirmatory factor analysis for applied research. New York, NY: Guilford Press (2015).
30. Hu, LT, and Bentler, PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. (1999) 6:1–55. doi: 10.1080/10705519909540118
31. Schermelleh-Engel, K, Moosbrugger, H, and Müller, H. Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods Psychol Res Online. (2003) 8:23–74. doi: 10.23668/psycharchives.12784
32. Chen, F, Curran, PJ, Bollen, KA, Kirby, J, and Paxton, P. An empirical evaluation of the use of fixed cutoff points in RMSEA test statistic in structural equation models. Sociol Methods Res. (2008) 36:462–94. doi: 10.1177/0049124108314720
33. Stevens, JP . Applied multivariate statistics for the social sciences. New York, NY: Routledge (2012).
34. Thurow, CF, Nunes, C, and Schneider, DR. Psychometrics and cultural adaptations of the communities that care youth survey: a systematic review. J Community Psychol. (2021) 49:2892–904. doi: 10.1002/jcop.22529
35. Briney, JS, Brown, EC, Hawkins, JD, and Arthur, MW. Predictive validity of established cut points for risk and protective factor scales from the communities that care youth survey. J Prim Prev. (2012) 33:249–58. doi: 10.1007/s10935-012-0280-1
36. Brook, J, Rifenbark, GG, Boulton, A, Little, TD, and McDonald, TP. Risk and protective factors for drug use among youth living in foster care. Child Adolesc Soc Work J. (2015) 32:155–65. doi: 10.1007/s10560-014-0345-5
37. Fagan, AA, Van Horn, ML, Hawkin, JD, and Arthur, MW. Gender similarities and differences in the association between risk and protective factors and selfreported serious delinquency. Prev Sci. (2007) 8:115–24. doi: 10.1007/s11121-006-0062-1
38. Feinberg, ME, Ridenour, TA, and Greenberg, MT. Aggregating indices of risk and protection for adolescent behavior problems: the communities that care youth survey. J Adolesc Health. (2007) 40:506–13. doi: 10.1016/j.jadohealth.2006.09.002
39. Glaser, RR, Horn, M, Arthur, MW, Hawkins, JD, and Catalano, RF. Measurement properties of the communities that care® youth survey across demographic groups. J Quant Criminol. (2005) 21:73–102. doi: 10.1007/s10940-004-1788-1
40. Guttmannova, K, Skinner, ML, Oesterle, S, White, HR, Catalano, RF, and Hawkins, JD. The interplay between marijuana-specific risk factors and marijuana use over the course of adolescence. Prev Sci. (2019) 20:235–45. doi: 10.1007/s11121-018-0882-9
41. Corrigan, MJ . The development of the ADSI: construct validity for the communities that care youth survey. J Child Adolesc Subst Abuse. (2014) 23:347–58. doi: 10.1080/1067828X.2014.928139
42. Groeger-Roth, F, Frisch, JU, Benit, N, and Soellner, R. Risikofaktoren für problematischen Substanzkonsum von Jugendlichen – Zur Anwendbarkeit des Communities That Care Schülersurveys auf kommunaler Ebene. SUCHT. (2015) 61:237–49. doi: 10.1024/0939-5911.a000379
43. Morojele, NK, Flisher, AJ, Muller, M, Ziervogel, CF, Reddy, P, and Lombard, CJ. Measurement of risk and protective factors for drug use and anti-social behavior among high school students in South Africa. J Drug Educ. (2002) 32:25–39. doi: 10.2190/MJDD-PC1G-4KUH-C1YW
44. Mejia-Trujillo, J, Perez-Gomez, A, and Reyes-Rodriguez, MF. Implementacion y adaptacion en Colombia del sistema preventivo communities that care (implementation and adaptation in Colombia of the communities that care). Adicciones. (2015) 27:253–64. doi: 10.20882/adicciones.750
45. Baheiraei, A, Soltani, F, Ebadi, A, Cheraghi, MA, Foroushani, AR, and Catalano, RF. Psychometric properties of the Iranian version of ‘communities that care youth survey’. Health Promot Int. (2014) 31:dau062–72. doi: 10.1093/heapro/dau062
46. Razali, MM, and Kliewer, W. Risk and protective factors for recreational and hard drug use among Malaysian adolescents and young adults. Addict Behav. (2015) 50:149–56. doi: 10.1016/j.addbeh.2015.06.022
47. Maguire, ER, Wells, W, and Katz, CM. Measuring community risk and protective factors for adolescent problem behaviors: evidence from a developing nation. J Res Crime Delinq. (2011) 48:594–620. doi: 10.1177/0022427810395148
48. Hawkins, JD, Oesterle, S, Brown, EC, Abbott, RD, and Catalano, RF. Youth problem behaviors 8 years after implementing the communities that care prevention system: a community-randomized trial. JAMA Pediatr. (2014) 168:122–9. doi: 10.1001/jamapediatrics.2013.4009
49. Shakeshaft, A, Doran, C, Petrie, D, Breen, C, Havard, A, Abudeen, A, et al. The effectiveness of community action in reducing risky alcohol consumption and harm: a cluster randomised controlled trial. PLoS Med. (2014) 11:e1001617–4. doi: 10.1371/journal.pmed.1001617
50. Runge, RA, Schlüter, H, Reder, M, and Soellner, R. Protective factors for adolescent problem behavior: direct or buffering effects? (2024). Manuscript in preparation.
51. Sameroff, A . Environmental risk factors in infancy. Pediatrics. (1998) 102:1287–92. doi: 10.1542/peds.102.SE1.1287
Keywords: confirmatory factor analysis, unidimensionality, risk factor, Communities That Care, validity, criterion validity, scale evaluation, adolescence
Citation: Reder M, Runge RA, Schlüter H and Soellner R (2024) The German Communities That Care Youth Survey: dimensionality and validity of risk factors. Front. Public Health. 12:1472347. doi: 10.3389/fpubh.2024.1472347
Edited by:
Allen C. Meadors, Independent Researcher, Seven Lakes, NC, United StatesReviewed by:
Jeff Bolles, Francis Marion University, United StatesKiara Lyons, Arizona State University West campus, United States
Copyright © 2024 Reder, Runge, Schlüter and Soellner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Maren Reder, rederm@uni-hildesheim.de