Neil Dagnall
Andrew Denovan
Kenneth Graham Drinkwater- 1Department of Psychology, Manchester Metropolitan University, Manchester, United Kingdom
- 2Department of Psychology, University of Huddersfield, Huddersfield, United Kingdom
The Revised Paranormal Belief Scale (RPBS) is the prevailing measure of supernatural credence. However, there exists only limited evidence to support the temporal stability and predictive validity of the instrument over time. Acknowledging this, the present study assessed the test–retest reliability of the RPBS using a large, heterogeneous sample across multiple trials. In addition, predictive validity was tested using a longitudinal statistical model, which focused on allied health outcomes (Perceived Stress and Somatic Complaints). A sample of 1,665 (Mage = 54.40, 853 females, 804 males, five non-binary and three not disclosing of gender) completed study measures at three time points separated by 2 month intervals. Prior to assessing temporal stability, assessment of structural validity and longitudinal invariance occurred. Test–retest reliability of the RPBS was in the moderate to high range across time intervals, and good internal consistency was observed. Furthermore, satisfactory stability coefficients existed for RPBS subfactors. Data-model fit for the predictive model was acceptable. Belief in the paranormal explained low variance over time in Perceived Stress and Somatic Complaints (between 2.4 and 4.2%). Findings supported the stability and reliability of the RPBS. In addition, they aligned with the notion that paranormal belief in the absence of high scores on cognitive-perceptual factors (e.g. transliminality and schizotypy), has a benign influence on perceived health.
Introduction
The Revised Paranormal Belief Scale (RPBS) (Tobacyk, 2004) is the most prevalently employed measure of supernatural credence (Drinkwater et al., 2017). The scale is a modified version of the Paranormal Belief Scale (PBS) (Tobacyk and Milford, 1983), which was created in response to intensified scholarly interest in paranormal phenomena. Central themes guiding development were novel forms of communication; existence of a founding universal principle; the notion of existence as comprising body, mind/spirit/soul and afterlife and the conceptualisation of reality as perceived rather than veridical (Rominger et al., 2022).
The synergy of phenomena encompassed by these themes encapsulated a breadth of beliefs, which were categorised as ‘religion, psi (clairvoyance, precognition, telepathy and psychokinesis), the occult, witchcraft, superstitions, the supernatural and extraordinary and extra-terrestrial life forms” (Tobacyk and Milford, 1983, p. 1025). Although other belief measures were in use at the time, these were constrained by assumptions about the nature of paranormal belief (Irwin, 2009; Dagnall et al., 2010a,b). Explicitly, content reflected preconceived views on dimensionality (unidimensional, Randall and Desrosiers, 1980 vs. multidimensional, Scheidt, 1973) rather than domain content. Noting this limitation, as part of the PBS development Tobacyk and Milford (1983) identified the structure of paranormal belief.
The PBS originated from analysis of 61 statements. These comprised items from existing instruments and newly constructed questions. Item selection was determined by criteria derived from current scientific understanding and orthodox notions of reality (Broad, 1953; Braude, 1978). The item pool was administered to psychology students (N = 391) at Louisiana Tech University. Analysis of scores identified seven orthogonal (uncorrelated) factors (i.e. Traditional Religious Beliefs, Psi, Witchcraft, Superstition, Spiritualism, Extraordinary Lifeforms and Precognition). Some researchers refer to these as the Big Orthogonal Seven model (BOS). Within the PBS, emergent factors were conceptually congruent; items reflected domain content and were internally consistent. The extracted 25-item scale employed a five-point Likert scale (1 = strongly disagree to 5 = strongly agree) and demonstrated psychometric integrity (convergent and discriminant validity). The BOS was important because it informed the content and structure of the subsequent revision, the 26-item RPBS (Tobacyk, 2004).
Modifications included enhancement of the Precognition subscale to ensure that items accurately evaluated construct domain, and amendments to the Extraordinary Lifeforms and Witchcraft subscales. To reduce range restriction, a seven-point Likert scale was adopted. These refinements improved subscale validity, reliability and Western cross-cultural standardisation. The importance of the RPBS is demonstrated by the fact that researchers later translated the scale into several languages (e.g. French, Bouvet et al., 2014; Spanish, Díaz-Vilela and Álvarez-González, 2004; Latvian, Utinans et al., 2015, Urdu, Rao et al., 2021 and Chinese, Shiah et al., 2010).
Investigators use the RPBS for several reasons. Firstly, the RPBS is an established instrument with attested psychometric properties. Secondly, the scale has featured prominently in published research (Dagnall et al., 2010b; Drinkwater, 2017). This signifies that it is a recognised, established measure of paranormal belief. Thirdly, since researchers have used the RPBS extensively it is possible to draw meaningful comparisons across studies. Fourthly, the RPBS in comparison to the Australian Sheep-Goat Scale (ASGS) (Thalbourne and Delin, 1993) assesses a wider range of construct content. The ASGS is another commonly used measure of paranormal belief. Social scientists tend to use the RPBS because of its breadth, whereas parapsychologists employ the ASGS because it focuses on core discipline domains (i.e. extra-sensory perception, psychokinesis and life after death) (Drinkwater et al., 2018a). Moreover, while the ASGS had been defined as a multidimensional instrument, Rasch analysis correcting for items displaying differential item functioning, recommends a single New Age Belief solution (Lange and Thalbourne, 2002) comprising only extra-sensory perception and psychokinesis.
Finally, the presence of subscales within the RPBS allows researchers to assess specific beliefs and/or examine whether belief type interacts differentially with psychological factors. Illustratively, the superstition subscale has featured as a standalone measure within published articles (see Wiseman and Watt, 2004; Dagnall et al., 2007, 2009). With reference to identifying psychological variations as a function of belief type, several studies report nuanced differences that were obscured by overall effects (e.g. Irwin et al., 2012a,b; Williams and Roberts, 2016; Kumar et al., 2020).
A caveat with the use of subscales is that alternative factorial solutions exist. Typical models are the original seven factors, a five-factor structure (Traditional Religious Belief, Psychic Beliefs, Superstition, Witchcraft and Anomalous Natural Phenomena) (Lawrence, 1995) and a two-factor purified model (New Age Philosophy, NAP and Traditional Paranormal Beliefs, TPB) (Lange et al., 2000). The latter is statistically rather than conceptually driven. Hence, emergent clusters combine subscale items and are defined in terms of function (i.e. influence over external events). NAP provides control at the individual/personal level, and TPB over social cultural factors. Despite correcting for differential item functioning, relatively few papers have used the two-factor in preference to the original RPBS solution (Drinkwater et al., 2018b). It is for this reason that the present paper focused on the stability of the seven factors and overall score over time. These solutions are the most frequently reported indices of paranormal belief (particularly the global score).
Notwithstanding debates about the factorial structure of the RBPS, there is currently only limited evidence to support the measure’s temporal stability. This issue is not specific to the RPBS as reliability across time, context and user are generally poorly reported within psychological literature (Aldridge et al., 2017). In this context, assessment of test–retest reliability is vitally important to establishing the psychometric integrity of the RPBS and determining the long-term effects of paranormal belief on health outcomes. Test–retest reliability is the systematic assessment of consistency, reproducibility and agreement between two or more respondent scores on the same measurement instrument under equivalent conditions. To demonstrate accuracy respondent scores must be similar across tests. This demonstrates that variations represent changes in the individual and are not the consequence of measurement inconsistency.
Test–retest reliability is also important because it complements validity by evidencing that a scale produces stable measurements. Hence, test–retest reliability is particularly important when evaluating the efficacy of treatments and interventions. In the current study, RPBS temporal stability was vital to determining the degree to which paranormal belief accurately predicted health-related outcomes over time.
A vital consideration when establishing test–retest is the gap between scale completions. The interval needs to be sufficient to evidence adequate stability, but not too long to result in significant attrition or opportunities for extrinsic factors to influence scores. Hence, with health measures intervals of 1–2 weeks are typically used (Polit, 2014). This is often constrained by practicality and hence, test–retest within validation papers often represents intervals of convenience (Watson, 2004).
In the case of the PBS, Tobacyk and Milford (1983) assessed the test–retest reliability of the PBS over a 4-week interval with a 25-subject sample and reported subscales correlations between 0.60 and 0.84. RPBS modification improved these figures (Tobacyk, 2004). Using a sample of 40 university students, 4-week test–retest reliabilities were Precognition 0.81, Witchcraft 0.93, Extraordinary Lifeforms 0.91, Traditional Religious Belief 0.95, Psi 0.71, Superstition 0.89 and Spiritualism 0.91. The overall scale test–retest was 0.92. While these figures indicate good to excellent test–retest reliability (Cicchetti, 1994; Fleiss, 2011), samples were restricted to students and the gap between testing points was only 4-weeks.
While this approach is typical within psychology, to effectively establish test–retest reliability, researchers need to consider the nature of the construct under observation and the extent to which they wish to extrapolate findings. This is vital since some psychological phenomena (e.g. mood and affect) fluctuate as a function of internal and external factors, whereas others (e.g. personality) remain relatively stable. Hence, paranormal belief is influenced by a combination of situational and dispositional factors. Additionally, belief types differ in lability. For example, traditional religious beliefs, which include notions of heaven and hell, are typically inculcated and enduring. While superstition varies in accordance with external factors that create negative affect (Padgett and Jorgenson, 1982; Dagnall et al., 2007). This aligns with the notion that superstitious belief and magical thinking serve as mechanisms for coping with environmental stress and uncertainty (Malinowski, 1948; Jahoda, 1969; Frost et al., 1993). Hence, these beliefs increase during periods of life pressure and unpredictability.
Noting these factors and the restricted nature of validation studies, it was necessary to further assess the test–retest reliability of the RBPS using a large, heterogeneous sample, across multiple time points over an extended period. This allowed the authors to thoroughly examine scale stability and provide further insights into the nature of belief. Explicitly, determine whether supernatural credence remained relatively unchanged over time or fluctuated in accordance with internal and external factors.
Paranormal belief and well-being
Recent research examining relationships between paranormal belief and well-being indicates that supernatural credence is benign in the absence of productive and disorganised cognitive-perceptual and psychopathological factors (Drinkwater et al., 2021; Dagnall et al., 2022a). It is the interaction with/and indirect influence of these factors, which explain negative belief-related outcomes. In the case of interactions, higher levels of belief combine with correlated constructs (i.e. schizotypy, transliminality and reality testing deficits) to create psychological profiles, which are characterised by a steady flow of unstructured mentation, and an overreliance on internal, intra-psychic activity and subjective interpretation. Psychological profiles with these characteristics are most strongly associated with lower well-being and poorer psychological adjustment.
In this context, believers comprise a set of subgroups and the effects of paranormal belief are defined by scores on concomitant cognitive-perceptual and psychopathological factors (Dagnall et al., 2011; Denovan et al., 2018). Hence, subgroups with higher psychopathology scores (schizotypy, depression and manic-depressive experience) tend to report lower well-being and poorer psychological adjustment. Subsequent work confirmed and extended these findings. Specifically, via a longitudinal design, Dagnall et al. (2022c) observed that over time the highest scoring psychopathology profile (vs. lower) was associated with higher negative and lower positive well-being.
With reference to indirect effects, within paranormal believers, transliminality and specific psychopathology-related variables in combination (i.e. the Unusual Experiences and Cognitive Disorganisation subscales of schizotypy and manic-depressive experience) predicted vulnerability to negative well-being outcomes (see Dagnall et al., 2022d). Transliminality is important because elevated levels indicate hypersensitivity to psychological material (both internal and external; Thalbourne, 1998). This suggests a reduction in the ability to attenuate and limit the stream of cognitive-perceptual information. It also potentially explains why transliminality is a prognosticator of psychopathology. Unusual Experiences reflect positive schizotypy (i.e. magical thinking, perceptual aberrations and hallucinations), and Cognitive Disorganisation thought disorder and additional allied aspects of psychosis (i.e. poor attention, decision-making and social anxiety). Manic-depressive experience is also important because higher scores reflect issues with affect regulation.
Correspondingly, using network analysis Dagnall et al. (2022b) observed that transliminality connected paranormal belief, positive schizotypy and psychopathology. Additionally, the association between transliminality and well-being was bridged by depressive experience. These findings indicate that transliminality embodies features such as lower cognitive flexibility (Peters et al., 1999) and heightened sensory information flow (i.e. reduced latent inhibition and hypersensitivity) (Carson, 2011), which are present in more extreme forms within psychosis (Waltz, 2017; Calvo et al., 2021). These shared features likely explain why transliminality correlates positively with supernatural credence and signifies vulnerability to psychopathology.
The present study
Acknowledging the limited work assessing the temporal stability of the RPBS, the present paper examined the test–retest reliability of the instrument with a large, heterogeneous sample across multiple time points. It was anticipated that this would establish temporal consistency and indicate whether the RPBS was similarly predictive of perceived stress (Cohen and Williamson, 1988) and somatic complaints (Gierk et al., 2014) at different intervals. These allied health outcomes were selected because previous research has reported relationships between paranormal belief and perceived stress (Lasikiewicz, 2016) and somatic complaints (Dagnall et al., 2022d; Laythe et al., 2022), and researchers generally acknowledge that stress and related psychosomatic symptoms are negatively associated with psychological and physical health (Wei et al., 2020).
Although causes of somatic symptoms are not currently well understood (Henningsen et al., 2007), researchers report that stress generally contributes to perceptions and interpretations of somatic symptoms (Witthöft and Hiller, 2010; Li et al., 2016). A model that potentially explains the relationship between stress and somatic symptoms is the stress-system model for functional somatic symptoms (Kozlowska, 2013). This views stress as a brain–body interaction underpinned by multiple interconnected systems. These include the hypothalamic–pituitary–adrenal axis, autonomic nervous system, immune-inflammatory system and brain stress systems allied to emotional states, salience detection and pain. Collectively, these protect individuals from threats. Since these function in a holistic, integrated manner activation of any system can initiate or dysregulate other systems. Accordingly, inappropriate or sustained activation of the stress system can affect its responsiveness and sensitivity, resulting in somatic symptoms (i.e. excessive awareness of physical sensations).
This manifests as persistent unspecific symptoms that cause individual concern and facilitate medical consultation, but are not classified as disease (Roenneberg et al., 2019). Moreover, stress-related somatic symptoms adversely affect emotional distress, which reciprocally via negative cognitive bias can increase reporting of somatic concerns (Wei et al., 2020). This model aligns with studies that report positive correlations between perceived stress and somatic complaints (e.g. Roenneberg et al., 2019). A further reason for selecting perceived stress and somatic complaints is that they are assessed over relatively short periods (the past month and 7-days respectively). Hence, scores are subject to variations in accordance with life pressures.
Commensurate with previous research, it was hypothesised that paranormal belief would remain stable over time and that supernatural natural credence, in the absence of other psychological factors (i.e. transliminality), would explain only low levels of variance in perceived stress and somatic complaints.
Materials and methods
Sample
This study used a longitudinal approach. Respondents (N = 1,665) completed study measures at three time points separated by two-month intervals. Sample mean age (Mage) was 54.40, range = 18–91, and gender was balanced, 853 females (Mage = 55.49, range = 18–91) and 804 males (Mage = 53.34, range = 18–86). Five participants were non-binary (Mage = 41.80, range = 23–69), and three preferred to not disclose gender (Mage = 48.66, range = 33–63). To participate, respondents had to be at least 18 years of age and free from a diagnosed mental illness.
Recruitment of participants occurred via Bilendi in 2020/2021, who source responses from a pool of individuals with an interest in participating in research studies. A large and representative sample was requested alongside a minimum age (18 years). Kees et al. (2017) report that online panel data are analogous to data collected via traditional means.
Measures
Revised Paranormal Belief Scale
The Revised Paranormal Belief Scale (RPBS; Tobacyk, 2004) assessed belief in the supernatural. The measure comprises 26-items, which are presented as statements (e.g. ‘Black Magic really exists’). Participants indicate their level of endorsement on a Likert scale (ranging from 1 = Strongly Disagree to 7 = Strongly Agree). Total scores range from 26 to 182 with higher scores representing greater paranormal belief. The RPBS comprises seven subscales: Traditional Religious Belief (4-items; life after death and heaven and hell), Psi (4-items; psychic powers), Witchcraft (4-items; casting skill and magical powers), Superstition (3-items; bad luck), Spiritualism (4-items; non corporeality), Extraordinary Lifeforms (3-items; existence of yet to be established entities) and Precognition (4-items, predicting future events). The RPBS overall and at the subscale level was internally reliable (see Table 1).
Table 1. Descriptives and reliability for RPBS, PSS, PSS factors, SSS-8 and RPBS factors.
Perceived Stress Scale
The Perceived Stress Scale (PSS-10; Cohen and Williamson, 1988) is a 10-item measure of individual perceptions of uncontrollability and unpredictability (life pressure) over the past month. Within the PSS-10 items appear as questions (e.g. ‘how often have you felt that you were on top of things?’), and respondents indicate their level of agreement by completing a 5-point Likert Scale (i.e. 0 = Never and 4 = Very Often). Two subscales frequently emerge from analyses of factor structure: PSS Distress (indexing feelings of distress) and PSS Coping (indexing ability to cope with problems) (Andreou et al., 2011). The PSS-10 has established psychometric properties (i.e. reliability and validity) (Denovan et al., 2019). After the computation of a total raw PSS-10 score, PSS Coping items were reverse-scored according to the scale author instructions (Cohen and Williamson, 1988). See Table 1 for internal reliability information.
Somatic Symptom Scale-8
The Somatic Symptom Scale-8 (SSS-8; Gierk et al., 2014) is an 8-item instrument that evaluates the presence and severity of somatic symptoms during the past 7-days. Problems appear in the form of symptoms (e.g. ‘Dizziness’) and respondents indicate occurrence/intensity on a 5-point Likert scale (i.e. 0 = Not at all and 4 = Very Much). The SSS-8 is an established, widely used, psychometrically robust measure (see Table 1 for study internal reliability) (Gierk et al., 2014).
Procedure
Respondents clicked a web link that directed them to the study information. This outlined the general purpose of the research project and the requirement to complete measures on three separate time points 2 months apart. Accordingly, respondents who agreed to participate provided an identification number, which allowed response matching across trials. Following the collation of the data sheet, the identification number was deleted. For respondents who progressed to the online measures, the procedure was the same at each time point. To reduce potential methodological issues, the investigators implemented procedural remedies. Firstly, to lessen potential common method variance, psychological distance was created between scales by accentuating construct uniqueness. Secondly, to reduce social desirability and evaluation apprehension, instructions stated that there were no correct or preferred responses. Finally, rotation of sections across respondents controlled for order effects. The demographic section was always completed first. Additionally, respondents were directed to take their time, complete measures at their own pace, carefully read items and complete all questions.
Ethics statement
The Faculty of Health, Psychology and Social Care Ethics Committee at Manchester Metropolitan University granted ethical approval (Project ID, 25390).
Analysis
Data screening examined normality. Then, internal reliability of the RPBS was evaluated prior to examining structural validity of the study scales (RPBS, PSS-10 and SSS-8) using confirmatory factor analysis (CFA). Subsequently, longitudinal invariance tests were performed to assess the notion that properties of the study scales did not significantly differ over time. Latent construct scores derived from the item-level analyses were implemented in a test of temporal (test–retest) reliability and predictive validity using path analysis. Path analysis employed a statistical model assessing the quantity of variance in perceived stress (PSS) and somatic complaints (SC) explained/predicted by paranormal belief (PB, as measured by the RPBS) over time.
In addition to inspection of explained variance, analysis evaluated the quality of model fit. This included consideration of a range of indices, specifically Confirmatory Fit Index (CFI), Standardized Root-Mean-Square Residual (SRMR) and Root-Mean-Squared Error of Approximation (RMSEA). Satisfactory thresholds for these are CFI > 0.90, SRMR <0.08 and RMSEA <0.06 (Hu and Bentler, 1999). These criteria were additionally employed when examining structural validity and invariance. For invariance, CFI differences ≤0.01 alongside RMSEA changes ≤0.015 are satisfactory (Chen, 2007).
Results
Data screening and internal reliability
Assessment of normality (Table 1) revealed that none of the study variables possessed skewness or kurtosis greater than the recommended threshold of +2 or-2 (Field and Miles, 2010). However, Mardia’s (1970) multivariate kurtosis was 63.51, suggesting non-normality. Accordingly, analysis employed the bootstrap method (with 1,000 resamples) to compute reliable standard errors and estimates (Byrne, 2010). Internal reliability of the RPBS was high at each time point (Time 1 α = 0.94, Time 2 α = 0.94, Time 3 α = 0.95). Internal reliability of the seven RPBS factors was satisfactory in all instances apart from Extraordinary Lifeforms. Concerns regarding this subscale in terms of its appropriateness and reliability have been raised previously (Drinkwater et al., 2017). The remaining factors demonstrated satisfactory reliability.
Structural validity
Prior to assessing latent relationships among the study constructs, structural validity involved examination of the RPBS (correlated seven-factor, higher-order seven-factor and a one-factor model as a null test), the SSS-8 (one-factor model) and the PSS-10 (correlated two-factor and a one-factor model as a null test). These conceptualisations of the study measures exist in the research literature (e.g. Tobacyk, 2004; Andreou et al., 2011; Denovan et al., 2018).
The correlated seven-factor RPBS solution demonstrated acceptable fit, χ2 (278, N = 1,665) = 2349.98, p < 0.001, CFI = 0.92, SRMR = 0.04, RMSEA = 0.06 (95% CI of 0.06 to 0.07). Similarly, the higher-order seven-factor model revealed satisfactory fit, χ2 (292, N = 1,665) = 2307.78, p < 0.001, CFI = 0.92, SRMR = 0.04, RMSEA = 0.06 (95% CI of 0.06 to 0.07). Fit was similar to the correlated solution. However, Akaike’s Information Criterion (a model comparison index) was marginally lower for the higher-order model (2505.78 vs. 2519.98), suggesting a more parsimonious solution. The one-factor model was unsatisfactory on all indices but SRMR, χ2 (299, N = 1,665) = 4639.75, p < 0.001, CFI = 0.83, SRMR = 0.05, RMSEA = 0.09 (95% CI of 0.09 to 0.10).
The two-factor correlated PSS-10 model evidenced good fit, χ2 (34, N = 1,665) = 294.23, p < 0.001, CFI = 0.97, SRMR = 0.03, RMSEA = 0.06 (95% CI of 0.06 to 0.07). However, a one-factor conceptualisation was unsatisfactory, χ2 (35, N = 1,665) = 2608.32, p < 0.001, CFI = 0.73, SRMR = 0.16, RMSEA = 0.21 (95% CI of 0.20 to 0.21). Conversely, a one-factor solution was suitable for the SSS-8 on all indices but RMSEA, χ2 (20, N = 1,665) = 457.84, p < 0.001, CFI = 0.92, SRMR = 0.04, RMSEA = 0.10 (95% CI of.10 to 0.12).
Longitudinal invariance
The first stage of longitudinal invariance testing involved assessing the baseline (configural) model for each construct. The RPBS demonstrated satisfactory invariance of form (Table 2). Progression from the test of form to factor loadings (metric) revealed no change in CFI, and a change of 0.001 in RMSEA. Constraining the intercepts (scalar level) additionally evidenced no CFI change alongside a change of 0.001 in RMSEA. Lastly, constraining residual variance to equality revealed a 0.002 CFI change and a 0.001 RMSEA change.
Table 2. Fit indices for longitudinal invariance models.
For the PSS-10, satisfactory configural invariance across time existed. No meaningful CFI difference existed at the metric level (factor loadings) with a 0.003 RMSEA difference. Similarly, the scalar level exhibited no CFI change and a 0.002 RMSEA difference. At the residual level, a 0.001 CFI change and a 0.003 RMSEA change occurred. The SSS-8 exhibited satisfactory fit at the configural level. Progression to the metric level revealed a 0.001 CFI difference and 0.005 RMSEA difference. The scalar model exhibited no CFI change and a 0.004 RMSEA difference. Lastly, the test of equal residuals revealed a 0.002 CFI difference and a 0.003 RMSEA change. These results indicated satisfactory invariance across the time intervals in this study for the RPBS, PSS-10 and SSS-8.
Test–retest reliability
The test–retest reliability coefficients (Pearson and Spearman’s Rho) were computed in relation to the latent construct scores derived from analysis of the statistical models. In contrast to total questionnaire scores, these are uncontaminated by measurement error (Lodder et al., 2022). Across the three time points for the RPBS (assessing paranormal belief, PB), test–retest was in the moderate to high range (> 0.70). Specifically, Time 1 and Time 2 r = 0.77, rs = 0.78, Time 1 and Time 3 r = 0.79, rs = 0.79, Time 2 and Time 3 r = 0.83, rs = 0.84. Additional scale and test–retest reliability coefficients appear in Table 3. Test–retest reliability was moderate to high across factors.
Table 3. Test–retest for RPBS, PSS, PSS factors, SSS-8 and RPBS factors using latent construct scores.
Internal construct validity
Scrutiny of associations at each time point using latent scores revealed small to moderate significant positive correlations between RPBS and PSS Distress (Time 1 = 0.20, Time 2 = 0.19, Time 3 = 0.19), and RPBS and SSS-8 (Time 1 = 0.19, Time 2 = 0.20, Time 3 = 0.20). Small significant negative correlations existed between RPBS and PSS Coping (Time 1 = −0.15, Time 2 = −0.19, Time 3 = −0.18).
Predictive validity
The fit of the PB - PSS and SC predictive model over time, using the latent factor scores, was satisfactory on all indices apart from RMSEA, χ2 (28, N = 1,665) = 1095.67, p < 0.001, CFI = 0.93, SRMR = 0.05, RMSEA = 0.15 (95% CI of 0.14 to 0.15). Inspection of modification indices revealed that a superior data-fit (and satisfactory RMSEA) occurred if error terms between Time 2 and Time 3 were allowed to correlate. However, this change conflicted with the depiction of temporal order within the analysis, and subsequently was not permitted.
Associations between variables at each time point (Figure 1) were significant and positive in all instances but for Time 2 PSS Distress and PSS Coping (r of −0.02), and Time 3 PB and PSS Distress (r of 0.05). In addition, paths between Time 1, Time 2 and Time 3 were significant for each respective construct (e.g. PB Time 1, Time 2, Time 3, etc.). Explained variance ranged from 38 to 69%. However, predictive relationships over time from PB to SC, PSS Distress and PSS Coping were not significant. Predictive paths were significant from PSS Distress to SC over time.
Figure 1. Predictive model of paranormal belief over time in relation to well-being (perceived stress and somatic complaints) using latent construct scores. Note. Standardised regression weights between variables are shown. Error is indicated by ‘e’. * p < 0.05, ** p < 0.001 using Bootstrapping significance estimates (1,000 resamples).
Accordingly, a model was specified in which paths from PSS to SC and PSS and SC sequential paths (e.g. PSS Time 1 to PSS Time 2, etc.) were fixed to zero to assess the quantity of explained variance in PSS Distress, PSS Coping and SC due to PB. This was low: 2.6 and 3.5% in PSS Distress at Time 2 and Time 3, 2.5 and 2.4% in PSS Coping at Time 2 and Time 3 and 3 and 4.2% in SC at Time 2 and Time 3. This indicated that a small proportion of variance in the outcomes over time was attributable to PB.
Discussion
The RPBS at both global and subscale levels demonstrated acceptable test–retest reliability (r > 0.70) (Del Rosario and White, 2005). This indicated that the RPBS possesses adequate temporal consistency and that belief in the paranormal remained stable across an extended time interval (6 months). Although the subscale correlations observed in the present study were lower than those reported by Tobacyk (2004), these conclusions are generally commensurate with the findings of the RPBS validation study.
The lower correlations within the present sample are attributable to the large heterogeneous sample used. In comparison, Tobacyk (2004) assessed test–retest over a 4-week period using a small, homogeneous sample of students. Despite this, variations in correlation size across intervals in this study remained within the acceptable range. Explicitly, Precognition 77 to 0.83, Witchcraft 0.75 to 0.82, Extraordinary Lifeforms 0.77 to 0.83, Traditional Religious Belief 0.76 to 82, Psi 0.74 to 0.80, Superstition 0.74 to 0.80 and Spiritualism 0.77 to 0.82. The overall scale test–retest was 0.77 to 0.83. Moreover, a longer period was employed in this study between each successive interval (2 months), and adequate stability coefficients additionally existed at 4 months (i.e. between Time 1 and 2).
In addition, longitudinal invariance tests revealed that the RPBS measurement properties did not significantly alter over time. This has not previously been established for the RPBS, and it indicated that any changes at the construct level were not contaminated by measurement bias (Lodder et al., 2022). Furthermore, a more accurate estimate of temporal stability existed in comparison with preceding research (e.g. Tobacyk, 2004) utilising sum scores (which risk contamination by measurement error), limiting the likelihood of under or overestimating stability over time.
Evidence of satisfactory stability overall supports the assumption that paranormal belief would remain relatively stable over time. Furthermore, a greater stability coefficient existed for subfactors including Traditional Religious Belief, with lower coefficients occurring for subfactors indexing superstition and magical thinking. This is consistent with the view that religious beliefs are typically enduring, whereas superstition and magical thinking are more likely to vary in accordance with situational factors (Padgett and Jorgenson, 1982; Dagnall et al., 2007).
Paranormal belief was similarly associated with outcome measures (PSS Distress, PSS Coping and Somatic Complaints) at each of the three assessments points, suggesting that the relationships were stable. Although most correlations were significant, they were small (r = −0.15 to.20, Gignac and Szodorai, 2016). Paranormal belief furthermore explained only 2.6 to 3.5% PSS Distress variance, 2.4 to 2.5% PSS Coping variance and 3 to 4.2% of the variance in Somatic Complaints over time. Essentially, the effects of paranormal belief on perceived stress and somatic complaints were weak in isolation, which supports the need to include additional variables to comprehend how this influences allied health outcomes/well-being. Indeed, these findings were consistent with recent work that has found that paranormal belief, in the context of negative well-being is typically benign (Dagnall et al., 2022a,b,c,d). Moreover, this supports the notion that adverse outcomes allied to paranormal belief arise from interactions with productive and disorganised cognitive-perceptual and psychopathological factors.
Limitations
Primary limitations of the study include the composition of the sample and the utilised measures. With regard to the sample, because this was sourced from a willing pool of respondents, there is the potential for self-selecting bias. Although this approach made pragmatic sense, a more naturally occurring sample would have produced findings that were increasingly representative of the population of interest. Relatedly, insufficient sample demographics were collected in relation to the study variables. Information including ethnicity, religious inclination and physical health would have been important to appreciate relative to paranormal belief and perceived stress and somatic complaints. As a minimum, this information should be controlled for in future research.
A second issue relates to the timing of the research. Data were collected in 2020/2021 surrounding the COVID-19 pandemic, and research indicates that increased levels of distress and stress existed during this period (e.g. Charles et al., 2021). Therefore, this should be appreciated in light of the findings. However, it is important to note that the sample reported perceived stress levels that were comparable to previously established norms with a general population sample (M = 14.20, SD = 6.20; Cohen, 1994).
In terms of the measures, the present study focused only on Perceived Stress and Somatic Complaints. Although robust scales were used, these represent a narrow conceptualisation of health. It would be important for future research to incorporate additional health-related features, such as the construct of psychological well-being. This could use the Psychological Well-Being Scale (Ryff and Keyes, 1995), which includes several domains including self-acceptance, purpose in life, autonomy and mastery. Moreover, recent research has established a link between paranormal belief and meaning in life (FioRito et al., 2021).
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Manchester Metropolitan University Faculty of Health, Psychology and Social Care Ethics Committee (December 2020; Project ID 25390). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements. Written informed consent was obtained from the individual (s) for the publication of any potentially identifiable images or data included in this article. The patients/participants provided their written informed consent to participate in this study.
Author contributions
AD and ND designed the study. ND provided conceptual input, developed the theoretical context, summarised findings, and edited all sections. AD collated measurement scales, arranged data collection, and performed analyses. KD and AD revised the final manuscript and prepared the draft submission. All authors contributed fully to the article and approved the final version.
Funding
We would like to thank the BIAL Foundation for their support of this project (grant number: 123/20).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Aldridge, V. K., Dovey, T. M., and Wade, A. (2017). Assessing test-retest reliability of psychological measures: persistent methodological problems. Eur. Psychol. 22, 207–218. doi: 10.1027/1016-9040/a000298
Andreou, E., Alexopoulos, E. C., Lionis, C., Varvogli, L., Gnardellis, C., Chrousos, G. P., et al. (2011). Perceived stress scale: reliability and validity study in Greece. Int. J. Environ. Res. Public Health 8, 3287–3298. doi: 10.3390/ijerph8083287
Bouvet, R., Djeriouat, H., Goutaudier, N., Py, J., and Chabrol, H. (2014). French validation of the revised paranormal belief scale. L'Encephale 40, 308–314. doi: 10.1016/j.encep.2014.01.004
Braude, S. (1978). “On the meaning of paranormal” in Philosophy and Parapsychology. ed. J. K. Ludwig (Prometheus Books), 227–244.
Broad, C. D. (1953). “The relevance of psychical research to philosophy” in Philosophy and Parapsychology. ed. J. K. Ludwig (Prometheus Books), 43–63.
Byrne, B. M. (2010). Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming. New York Routledge/Taylor & Francis Group.
Calvo, E. M., Ered, A., Maxwell, S. D., and Ellman, L. M. (2021). Behavioural inhibition system sensitivity is no longer associated with psychotic-like experiences after controlling for depression and anxiety symptoms. Early Interv. Psychiatry 15, 1217–1223. doi: 10.1111/eip.13067
Carson, S. H. (2011). Creativity and psychopathology: a shared vulnerability model. Can. J. Psychiatry 56, 144–153. doi: 10.1177/070674371105600304
Charles, N. E., Strong, S. J., Burns, L. C., Bullerjahn, M. R., and Serafine, K. M. (2021). Increased mood disorder symptoms, perceived stress, and alcohol use among college students during the COVID-19 pandemic. Psychiatry Res. 296:113706. doi: 10.1016/j.psychres.2021.113706
Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Model. Multidiscip. J. 14, 464–504. doi: 10.1080/10705510701301834
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychol. Assess. 6, 284–290. doi: 10.1037/1040-3590.6.4.284
Cohen, S. (1994). Perceived Stress Scale - Mind Garden. Available online at: https://www.mindgarden.com/documents/PerceivedStressScale.pdf (Accessed December 12, 2022).
Cohen, S., and Williamson, G. M. (1988). “Perceived stress in a probability sample in the United States” in The Social Psychology of Health. eds. S. Spacapan and S. Oskamp (Newbury Park, CA: Sage), 31–67.
Dagnall, N., Denovan, A., and Drinkwater, K. G. (2022a). Variations in well-being as a function of paranormal belief and psychopathological symptoms: a latent profile analysis. Front. Psychol. 13:886369. doi: 10.3389/fpsyg.2022.886369
Dagnall, N., Denovan, A., and Drinkwater, K. G. (2022b). Paranormal belief, cognitive-perceptual factors, and well-being: a network analysis. Front. Psychol. 13:967823. doi: 10.3389/fpsyg.2022.967823
Dagnall, N., Denovan, A., and Drinkwater, K. G. (2022c). A latent profile analysis and longitudinal assessment of the relationship between paranormal belief, psychopathological symptoms, and well-being. Front. Psychol., 1–35.
Dagnall, N., Denovan, A., Drinkwater, K. G., and Escolà-Gascón, Á. (2022d). Paranormal belief and well-being: the moderating roles of transliminality and psychopathology-related facets. Front. Psychol. 13:915860. doi: 10.3389/fpsyg.2022.915860
Dagnall, N., Drinkwater, K., and Parker, A. (2011). Alien visitation, extra-terrestrial life, and paranormal beliefs. J. Sci. Explorat. 25, 699–720.
Dagnall, N., Munley, G., Parker, A., and Drinkwater, K. (2010a). The relationship between belief in extra-terrestrial life, UFO-related beliefs and paranormal belief. J. Soc. Psychical Res. 74, 1–14.
Dagnall, N., Parker, A., and Munley, G. (2007). Superstitious belief-negative and positive superstitions and psychological functioning. Eur. J. Parapsychol. 22, 121–137.
Dagnall, N., Parker, A., and Munley, G. (2009). Assessing superstitious belief. Psychol. Rep. 104, 447–454. doi: 10.2466/PR0.104.2.447-454
Dagnall, N., Parker, A., Munley, G., and Drinkwater, K. (2010b). Common paranormal belief dimensions. J. Sci. Explorat. 24, 431–447.
Del Rosario, P. M., and White, R. M. (2005). The narcissistic personality inventory: test–retest stability and internal consistency. Personal. Individ. Differ. 39, 1075–1081. doi: 10.1016/j.paid.2005.08.001
Denovan, A., Dagnall, N., Dhingra, K., and Grogan, S. (2019). Evaluating the perceived stress scale among UK university students: implications for stress measurement and management. Stud. High. Educ. 44, 120–133. doi: 10.1080/03075079.2017.1340445
Denovan, A., Dagnall, N., Drinkwater, K., and Parker, A. (2018). Latent profile analysis of schizotypy and paranormal belief: associations with probabilistic reasoning performance. Front. Psychol. 9:35. doi: 10.3389/fpsyg.2018.00035
Díaz-Vilela, L., and Álvarez-González, C. J. (2004). Differences in paranormal beliefs across fields of study from a Spanish adaptation of Tobacyk's RPBS. J. Parapsychol. 68, 405–422.
Drinkwater, K. G. (2017). Belief in the Paranormal: Measurement Development and Evaluation. doctoral dissertation. Manchester Manchester Metropolitan University.
Drinkwater, K., Dagnall, N., Denovan, A., Parker, A., and Clough, P. (2018b). Predictors and associates of problem–reaction–solution: statistical bias, emotion-based reasoning, and belief in the paranormal. SAGE Open 8. doi: 10.1177/2158244018762999
Drinkwater, K. G., Dagnall, N., Denovan, A., and Williams, C. (2021). Paranormal belief, thinking style and delusion formation: a latent profile analysis of within-individual variations in experience-based paranormal facets. Front. Psychol. 12:670959. doi: 10.3389/fpsyg.2021.670959
Drinkwater, K., Denovan, A., Dagnall, N., and Parker, A. (2017). An assessment of the dimensionality and factorial structure of the revised paranormal belief scale. Front. Psychol. 8:1693. doi: 10.3389/fpsyg.2017.01693
Drinkwater, K., Denovan, A., Dagnall, N., and Parker, A. (2018a). The Australian sheep-goat scale: an evaluation of factor structure and convergent validity. Front. Psychol. 9:1594. doi: 10.3389/fpsyg.2018.01594
FioRito, T. A., Abeyta, A. A., and Routledge, C. (2021). Religion, paranormal beliefs, and meaning in life. Relig. Brain Behav. 11, 139–146. doi: 10.1080/2153599X.2020.1824938
Frost, R. O., Krause, M. S., McMahon, M. J., Peppe, J., Evans, M., McPhee, A. E., et al. (1993). Compulsivity and superstitiousness. Behav. Res. Ther. 31, 423–425. doi: 10.1016/0005-7967(93)90101-Y
Gierk, B., Kohlmann, S., Kroenke, K., Spangenberg, L., Zenger, M., Brähler, E., et al. (2014). The somatic symptom scale–8 (SSS-8). JAMA Intern. Med. 174, 399–407. doi: 10.1001/jamainternmed.2013.12179
Gignac, G. E., and Szodorai, E. T. (2016). Effect size guidelines for individual differences researchers. Personal. Individ. Differ. 102, 74–78. doi: 10.1016/j.paid.2016.06.069
Henningsen, P., Zipfel, S., and Herzog, W. (2007). Management of functional somatic syndromes. Lancet 369, 946–955. doi: 10.1016/S0140-6736(07)60159-7
Hu, L. T., and Bentler, P. M. (1999). Cut off criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 6, 1–55. doi: 10.1080/10705519909540118
Irwin, H. J. (2009). The Psychology of Paranormal Belief: A Researcher's Handbook. Hatfield University of Hertfordshire Press.
Irwin, H. J., Dagnall, N., and Drinkwater, K. (2012a). Paranormal beliefs and cognitive processes underlying the formation of delusions. Austr. J. Parapsychol. 12, 107–126.
Irwin, H. J., Dagnall, N., and Drinkwater, K. (2012b). Paranormal belief and biases in reasoning underlying the formation of delusions. Aust. J. Parapsychol. 12, 7–21.
Kees, J., Berry, C., Burton, S., and Sheehan, K. (2017). An analysis of data quality: Professional panels, student subject pools, and Amazon’s Mechanical Turk. J. Advert. 46, 141–155. doi: 10.1080/00913367.2016.1269304
Kozlowska, K. (2013). Functional somatic symptoms in childhood and adolescence. Curr. Opin. Psychiatry 26, 485–492. doi: 10.1097/YCO.0b013e3283642ca0
Kumar, P., Saini, S. S., Sharma, R., and Kumar, K. (2020). Personality and paranormal belief: a study among university students. J. Psychosoc. Res. 15, 651–660. doi: 10.32381/JPR.2020.15.02.26
Lange, R., Irwin, H. J., and Houran, J. (2000). Top-down purification of Tobacyk’s revised paranormal belief scale. Personal. Individ. Differ. 29, 131–156. doi: 10.1016/S0191-8869(99)00183-X
Lange, R., and Thalbourne, M. A. (2002). Rasch scaling paranormal belief and experience: structure and semantics of Thalbourne's Australian sheep-goat scale. Psychol. Rep. 91, 1065–1073. doi: 10.2466/pr0.2002.91.3f.1065
Lasikiewicz, N. (2016). Perceived stress, thinking style, and paranormal belief. Imagin. Cogn. Pers. 35, 306–320. doi: 10.1177/0276236615595235
Lawrence, T. R. (1995). How many factors of paranormal belief are there? A critique of the paranormal belief scale. J. Parapsychol. 59, 3–25.
Laythe, B., Houran, J., Dagnall, N., Drinkwater, K., and O’Keeffe, (2022). Ghosted! Exploring the Haunting Reality of Paranormal Encounters. McFarland.
Li, J., Ding, H., Han, W., Jin, L., Kong, L. N., Mao, K. N., et al. (2016). The association of work stress with somatic symptoms in Chinese working women: a large cross-sectional survey. J. Psychosom. Res. 89, 7–10. doi: 10.1016/j.jpsychores.2016.08.001
Lodder, P., Kupper, N., Mols, F., Emons, W. H., and Wicherts, J. M. (2022). Assessing the temporal stability of psychological constructs: an illustration of type D personality, anxiety and depression. J. Res. Pers. 101:104299. doi: 10.1016/j.jrp.2022.104299
Mardia, K. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika 57, 519–530. doi: 10.1093/biomet/57.3.519
Padgett, V. R., and Jorgenson, D. O. (1982). Superstition and economic threat: Germany, 1918–1940. Personal. Soc. Psychol. Bull. 8, 736–741. doi: 10.1177/0146167282084021
Peters, E., Day, S., McKenna, J., and Orbach, G. (1999). Delusional ideation in religious and psychotic populations. Br. J. Clin. Psychol. 38, 83–96. doi: 10.1348/014466599162683
Polit, D. F. (2014). Getting serious about test–retest reliability: a critique of retest research and some recommendations. Qual. Life Res. 23, 1713–1720. doi: 10.1007/s11136-014-0632-9
Randall, T., and Desrosiers, M. (1980). Measurement of supernatural belief: sex differences and locus of control. J. Pers. Assess. 44, 493–498. doi: 10.1207/s15327752jpa4405_9
Rao, W., Ghayas, S., and Awan, S. M. (2021). Urdu translation, adaptation, and validation of revised paranormal belief scale for Pakistani people. Mental Health Relig. Cult. 24, 306–315. doi: 10.1080/13674676.2018.1489376
Roenneberg, C., Sattel, H., Schaefert, R., Henningsen, P., and Hausteiner-Wiehle, C. (2019). Functional somatic symptoms. Dtsch. Arztebl. Int. 116, 553–560. doi: 10.3238/arztebl.2019.0553
Rominger, C., Fink, A., Perchtold-Stefan, C. M., Schulter, G., Weiss, E. M., and Papousek, I. (2022). Creative, yet not unique? Paranormal belief, but not self-rated creative ideation behavior is associated with a higher propensity to perceive unique meanings in randomness. Heliyon 8:e09269. doi: 10.1016/j.heliyon.2022.e09269
Ryff, C. D., and Keyes, C. L. M. (1995). The structure of psychological well-being revisited. J. Pers. Soc. Psychol. 69, 719–727. doi: 10.1037/0022-3514.69.4.719
Scheidt, R. (1973). Belief in supernatural phenomena and locus of control. Psychol. Rep. 32, 1159–1162. doi: 10.2466/pr0.1973.32.3c.1159
Shiah, Y. J., Wu, M. H., Tam, W. C. C., and Chang, F. (2010). Paranormal beliefs and religiosity: Chinese version of the revised paranormal belief scale. Psychol. Rep. 107, 367–382. doi: 10.2466/08.09.17.PR0.107.5.367-382
Thalbourne, M. A. (1998). Transliminality: further correlates and a short measure. J. Am. Soc. Psych. Res. 92, 402–419.
Thalbourne, M. A., and Delin, P. S. (1993). A new instrument for measuring the sheep-goat variable: its psychometric properties and factor structure. J. Soc. Psych. Res. 59, 172–186.
Tobacyk, J. J. (2004). A revised paranormal belief scale. Int. J. Transpers. Stud. 23, 94–98. doi: 10.24972/ijts.2004.23.1.94
Tobacyk, J., and Milford, G. (1983). Belief in paranormal phenomena: assessment instrument development and implications for personality functioning. J. Pers. Soc. Psychol. 44, 1029–1037. doi: 10.1037/0022-3514.44.5.1029
Utinans, A., Ancane, G., Tobacyk, J. J., Boyraz, G., Livingston, M. M., and Tobacyk, J. S. (2015). Paranormal beliefs of Latvian college students: a Latvian version of the revised paranormal belief scale. Psychol. Rep. 116, 116–126. doi: 10.2466/08.17.PR0.116k14w9
Waltz, J. A. (2017). The neural underpinnings of cognitive flexibility and their disruption in psychotic illness. Neuroscience 345, 203–217. doi: 10.1016/j.neuroscience.2016.06.005
Watson, D. (2004). Stability versus change, dependability versus error: issues in the assessment of personality over time. J. Res. Pers. 38, 319–350. doi: 10.1016/j.jrp.2004.03.001
Wei, D., Liu, Y., Zhuang, K., Lv, J., Meng, J., Sun, J., et al. (2020). Brain structures associated with individual differences in somatic symptoms and emotional distress in a healthy sample. Front. Hum. Neurosci. 14:492990. doi: 10.3389/fnhum.2020.492990
Williams, E., and Roberts, B. L. (2016). The relationship between paranormal belief and the HEXACO domains of personality. J. Empir. Theol. 29, 212–238. doi: 10.1163/15709256-12341341
Wiseman, R., and Watt, C. (2004). Measuring superstitious belief: why lucky charms matter. Personal. Individ. Differ. 37, 1533–1541. doi: 10.1016/j.paid.2004.02.009
Keywords: longitudinal assessment, predictive validity, Revised Paranormal Belief Scale, test–retest reliability, health outcomes
Citation: Dagnall N, Denovan A and Drinkwater KG (2023) Longitudinal assessment of the temporal stability and predictive validity of the Revised Paranormal Belief Scale. Front. Psychol. 13:1094701. doi: 10.3389/fpsyg.2022.1094701
Edited by:
Douglas A. MacDonald, University of Detroit Mercy, United StatesReviewed by:
Sandor Rozsa, Károli Gáspár University of the Reformed Church in Hungary, HungaryCristian Ramos-Vera, Cesar Vallejo University, Peru
Copyright © 2023 Dagnall, Denovan and Drinkwater. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Neil Dagnall, ✉ bi5kYWduYWxsQG1tdS5hYy51aw==