Kieran T. Sullivan
Kathryn Bruchmann- Department of Psychology, Santa Clara University, Santa Clara, CA, United States
Objective: To test the reliability and validity of the Online Jealousy Scale.
Background: Romantic jealousy is often examined in online and social media settings and a validated measure of online jealousy is needed.
Method: Across two studies, the present research tests the psychometric properties of the Online Jealousy Scale (adapted from the Facebook Jealousy Scale) on an undergraduate (Study 1, N = 111) and two broader community (Study 2, N = 200; Study 3, N = 143) samples.
Results: Data across all studies provide evidence of strong inter-item and test–retest reliability; and construct, convergent and discriminant validity. Consistent with other jealousy measures, evidence of three factors emerged: emotional, cognitive, and behavioral jealousy.
Discussion: These studies indicate that the OJS is a reliable and valid instrument.
Implications: This measure fills the need for a valid, reliable assessment of online jealousy and can be used in research about online jealousy across age and relationship type. It may also be useful for individual or couple therapy.
General introduction
Online interactions (via social networking sites, direct messaging, etc.) provide rich fodder for romantic jealousy (Rus and Tiemensma, 2017; Muise et al., 2009), ranging from exposure to ambiguous information about a partner (e.g., a picture of a partner’s arm around a member of the preferred sex) to misinterpretation of direct messaging (e.g., unarticulated “rules” about response time, absence of nonverbal cues, etc.). Research examining jealousy related to online interactions has shed light on the processes, predictors, and impacts of jealousy in the digital arena. This work is important, as online jealousy is associated with negative relationship outcomes, including an increased risk of relationship aggression (Demirtaş-Madran, 2018; Quiroz et al., 2024).
This growing body of research is hampered, however, by limitations in the measurement of online jealousy, such as single-item assessments and assessments using measures untested in the digital arena (e.g., the Multidimensional Jealousy Scale; Pfeiffer and Wong, 1989). The Facebook Jealousy Scale (Del Sánchez-Fuentes et al., 2023) was developed to address these limitations but is itself limited by the specificity of the platform and the paucity of psychometric evidence. Indeed, in their review of social media and relationships Rus and Tiemensma (2017) identified further validation of the Facebook Jealousy Scale as “of particular priority” (p. 701) for moving the field forward.
Developing a measure that works across platforms is particularly important given the wide use of alternate sites such as Instagram or Tiktok (Auxier and Anderson, 2021; Vogels et al., 2022) particularly by young adults – a key demographic when examining romantic jealousy related to online interactions (Social Media Fact Sheet, 2021). Thus, the purpose of the present study is to adapt and extend the FJS to accommodate various platforms, and to assess the reliability and validity of the resulting Online Jealousy Scale (OJS) across several samples.1
The nomological network of online jealousy
Jealousy construct
Jealousy is a complex, multidimensional experience that involves emotions (e.g., anger, fear), cognitions (e.g., worry or suspicions), and behaviors (e.g., surveillance of partner; Pfeiffer and Wong, 1989). While the FJS (Muise et al., 2009) is typically used as a single, unitary construct, the items on the FJS are readily categorized as emotional (e.g., “How likely are you to be upset if your partner does not post an accurate relationship status on Facebook?), cognitive (e.g., “How likely are you to worry that your partner will become romantically involved with someone on Facebook?”), and behavioral (e.g., “How likely are you to monitor your partner’s activities on Facebook?”). Distinguishing these facets is important because they are differentially associated with relationship variables; for example, Dandurand and LaFontaine (2014) found that relationship satisfaction was negatively correlated with cognitive jealousy, positively correlated with emotional jealousy, and was not related to behavioral jealousy after accounting for cognitive jealousy. In the following, we will summarize previous findings regarding the demographic differences in online jealousy, the constructs we expect to be related to online jealousy (convergent validity), and the constructs we expect to be only weakly related or not related to online jealousy (discriminant validity).
Demographic differences in online jealousy
Gender
Women experience higher levels of online jealousy (e.g., Marshall et al., 2013; Muise et al., 2009; Rus and Tiemensma, 2017; Wang et al., 2024; cf., Demirtaş-Madran, 2018) than men. Women report higher levels of FB-evoked feelings of jealousy and jealousy-motivated behaviors (McAndrew and Shah, 2013) and are more likely to monitor their partner’s FB activities (Muise et al., 2014).
Age
While the preponderous of studies use young adult samples (Rus and Tiemensma, 2017), findings from studies with more heterogenous samples indicate that online jealousy decreases as people age (Demirtaş-Madran, 2018), which is consistent with studies of age and offline jealousy generally (Lantagne and Furman, 2017).
Convergent validity
Interpersonal electronic surveillance
Interpersonal electronic surveillance involves the use of digital technology to monitor partners’ behavior (Tokunaga, 2011). Surveillance is a predictor of (Elphinston and Noller, 2011; Imperato et al., 2023; Muise et al., 2009) and antecedent to (Marshall et al., 2013; Muise et al., 2014) online jealousy. Surveillance most strongly overlaps with the behavioral dimension of jealousy, and items from the FJS (Muise et al., 2009) that explicitly address partner monitoring have been used to assess surveillance in past studies (e.g., Elphinston and Noller, 2011; Utz and Beukeboom, 2011).
Attachment anxiety
Attachment anxiety involves low self-worth, preoccupation with fears of abandonment, and the need for constant reassurance by intimate partners. Attachment anxiety is associated with higher FB jealousy (Marshall et al., 2013), stronger jealous reactions to viewing touch between a romantic partner and a friend of the preferred sex online (Miller et al., 2014), more surveillance of romantic partners (Fox and Warber, 2014; Marshall et al., 2013; Muise et al., 2014), and higher likelihood of engaging in jealousy induction (Wegner et al., 2018). Although studies of the association between attachment anxiety and specific facets of online jealousy are scarce, studies of multidimensional jealousy suggest that attachment anxiety is more strongly correlated with cognitive and behavioral jealousy, compared to emotional jealousy (e.g., Bevan, 2017; Frampton, 2024; Rodriguez et al., 2015; Rydell and Bringle, 2007).
Discriminant validity
Social comparison
Social comparisons with others often result in feelings of envy (e.g., Smith et al., 2008; Appel et al., 2016), which is conceptually similar but distinct from romantic jealousy. Envy can have a negative impact on well-being or can inspire and motivate behavior (e.g., Meier and Schafer, 2018; Tandoc and Goh, 2023) whereas romantic jealousy is likely to lead to more negative emotions such as anger and sadness (Salovey and Rodin, 1986).
Narcissism
Narcissism has some overlap with romantic jealousy (e.g., people high in narcissism are more likely to try to induce jealousy; Tortoriello et al., 2017), but is only weakly related to doubts or jealousy towards partners (e.g., Foster and Campbell, 2005). Additionally, grandiose narcissism is related to social media usage in that individuals high in grandiose narcissism are more likely to post on social media, have more followers, post more self-promoting content, and post more photos of themselves (e.g., Buffardi and Campbell, 2008; also see McCain and Campbell, 2018 for a meta-analysis).
Relationship satisfaction
Associations between online jealousy and relationship satisfaction are generally negative and weak (e.g., Bevan, 2018; Telli and Yavuz Güler, 2023), this holds true for the FJS emotional and behavioral subscales. The cognitive subscale is more strongly associated with relationship satisfaction, however (e.g., DiBello et al., 2015).
Investment in relationship
The investment model posits that relationships endure when couples are strongly committed to and invested in their relationship. Investment includes such things as the amount of time spent in the relationship and personal disclosure to their partner (Kelley and Thibaut, 1978). Investment and jealousy, while both related to relationship satisfaction (e.g., Tokunaga, 2016), are not associated or only very weakly associated with one another (Bevan, 2017; Drouin et al., 2014; Tokunaga, 2016). Cognitive jealousy is the most likely to have a positive association with investment (Bevan, 2008; Bevan, 2017).
Study 1
The purpose of Study 1 was threefold: first, to generate an item pool using questions adapted from the Facebook Jealousy Scale (FJS; Muise et al., 2009) and newly created questions designed to assess all types of online interactions and social media platforms; second, to use item analysis to identify optimal items for the new Online Jealousy Scale (OJS); and third, to test the reliability and validity of the OJS. Validity was assessed by examining the associations between the OJS (and its subscales) and measures of other constructs in the nomological network of online jealousy. If the OJS is a valid measure of online jealousy, scores should correlate strongly with other measures of jealousy, moderately with closely related constructs (i.e., interpersonal surveillance, attachment anxiety), and relatively weakly with more distantly related constructs (i.e., social comparison, narcissism, relationship satisfaction, and relationship investment). Further, among the subscales, the OJS behavioral scale should have the strongest correlation with interpersonal surveillance; the OJS cognitive and behavioral subscales should have the strongest correlations with attachment anxiety; and the OJS cognitive subscale should have the highest correlations with relationship satisfaction and investment.
Scale development
Items from the Facebook Jealousy scale (FJS, Muise et al., 2009) were updated (e.g., changing “opposite sex” to “preferred sex”) and adapted to assess jealousy across social media platforms. For example, ‘How likely are you to monitor your partner’s activities on Facebook?” was changed to “How likely are you to monitor your partner’s social media activity?” and “How likely are you to become jealous after seeing that your partner has posted a message on the wall of someone of the opposite sex” was changed to “How upset would you be if your partner posted a message to someone of the preferred sex using social media.”
Next, eight research assistants (two teams of four) and the principal investigator brainstormed additional items for each of the three subscales to ensure that the questions reflected a variety of online interactions beyond social media. Examples include “How upset would you be if your partner did not respond to your personal messages right away?” (emotional subscale), “How likely are you to suspect that your partner is deliberately keeping you from seeing what is on their device?” (cognitive subscale), and “How likely are you to insist that your partner post pictures of you on social media?” (behavioral subscale). After each group came up with potential items, the two groups compared and discussed items; in total, 24 new items were added to the scale (for a total of 51 items).
Method
Participants and procedures
General Psychology students who were in a relationship (N = 128) participated in exchange for partial course credit. See Table 1 for demographic information including gender, ethnicity, age, income, and education for all studies. Institutional review board approval was obtained before beginning the study. Participants completed an online survey including a consent form, demographic questions, and study measures. One week later, 61% of the participants (n = 66) completed the OJS.
Table 1. Sample characteristics for Studies 1, 2, and 3.
As a quality control measure, the final item for the initial survey and the follow-up survey read “One last thing: Our knowledge about the effects of social media on dating relationships will only be as good as the data we collect. Please let us know how much you were able to pay attention to and consider each question.” Participants responded on a 5-point scale ranging from 1 (Very much) to 5 (Not at all); data from participants who selected 4 (a little bit) or 5 (17 out of 128 participants) were not included in the study, yielding a final sample size of 111.
Major change
At Time 2, participants were asked if they had experienced a major change or event in their relationship since participating in the first part of the study. Respondents were asked to select all options that related to them, including a major fight, significant jealousy, significant decrease in certainty about their relationship, and/or if they broke up. Respondents could also select “other” and describe a major change. Thirteen participants reported a serious, negative relationship event between Time 1 and Time 2; these participants were excluded from test–retest analyses.
Validity measures
Jealousy
Jealousy was measured using the Multidimensional Jealousy Scale (MJS; Pfeiffer and Wong, 1989), a widely used, well-validated measure of jealousy. The MJS is comprised of three subscales with 8 items each that measure emotional (e.g., “How upset would you be if your partner comments to you on how great looking a particular member of the preferred sex is”), cognitive (e.g., “How often do you … suspect that your partner may be attracted to someone else”), and behavioral jealousy (e.g., “How often do you … pay your partner a surprise visit just to see who is with him/her”). Participants respond on a scale of 1 (not at all upset) to 7 (extremely upset) for the emotional subscale, 1 (never) to 7 (all the time) for the cognitive subscale, and 1 (never) to 7 (extremely often) for the behavior subscale. Cronbach’s alpha in the current study for the full scale (α = 0.90), emotional subscale (α = 0.90), cognitive subscale (α = 0.84), and behavioral subscale (α = 0.83) indicated that the scale was reliable.
Interpersonal electronic surveillance
Interpersonal electronic surveillance was assessed using the Interpersonal Electronic Surveillance Scale (IESS; Tokunaga, 2016), a 7-item scale that includes questions such as “I often monitor my partner’s social networking site” and “I check up on my partner through updates on his or her social networking site.” Participants responded on a scale of 1 (strongly disagree) to 7 (strongly agree). Coefficient alpha analysis revealed that the scale was reliable (α = 0.94).
Attachment anxiety
Attachment anxiety was assessed using the attachment anxiety subscale of the Experiences in Close Relationship Scale-Short Form (6 items; Wei et al., 2007). Participants responded on a scale of 1 (strongly disagree) to 5 (strongly agree). Cronbach’s alpha for attachment anxiety (α = 0.73) indicated that the scale was reliable.
Social comparison
Social comparison orientation was assessed using the 11-item Iowa-Netherlands Comparison Orientation Measure (INCOM; Gibbons and Buunk, 1999). Participants rated their agreement with items such as “I always pay a lot of attention to how I do things compared with how others do things” and “I often compare how I am doing socially (e.g., social skills, popularity) with other people.” Participants responded on a scale of 1 (strongly disagree) to 5 (strongly agree). Cronbach’s alpha in the current study (α = 0.85) indicated that the scale was reliable.
Narcissism
Narcissism was measured using the 52-item Pathological Narcissism Inventory (PNI; Pincus, 2013). Participants indicated their agreement with questions assessing Narcissism (e.g., I often fantasize about performing heroic deeds). Participants responded on a scale of 1 (not at all like me) to 5 (very much like me). Cronbach’s alpha in the current study (α = 0.96) indicated that the scale was reliable.
Relationship satisfaction
Relationship satisfaction was assessed using The Couples Satisfaction Index (CSI-8, Funk and Rogge, 2007), a commonly used measure of relationship satisfaction that includes questions such as “Please indicate the degree of happiness, all things considered, of your relationship.” Coefficient alpha for this scale was 0.99.
Investment
Investment was measured using the investment size and commitment subscales from the Rusbult Investment Level Scale (Rusbult et al., 1998), a 17-item scale that includes questions such as “I have put a great deal into our relationship that I would lose if the relationship were to end” (investment size) and “I want our relationship to last for a very long time” (commitment). Participants responded on a scale of 1 (strongly disagree) to 7 (strongly agree). Cronbach’s alpha (α = 0.94) indicated that the scale was reliable.
Results
Item analysis
To identify optimal items from the pool, a principal component factor analysis with varimax rotation, restricted to three factors, was conducted. Items with factor loadings below 0.50 (9 items) and items that loaded on two factors (three items) were eliminated. The remaining items loaded on the expected factors; factor loadings ranged from 0.51 to 0.89 (M = 0.81; emotion factor), from 0.72 to 0.80 (M = 0.77; cognitive factor), and from 0.83 to 0.94 (M = 0.90; behavioral factor). The emotional factor accounted for 39.91% of the variance, the behavioral factor accounted for an additional 9.85%, and the cognitive factor accounted for an additional 7.12%, with a total of 48.87% of the variance accounted for.
To identify optimal items from the pool, items with factor loadings below 0.50 were eliminated (9 items). Next, item endorsement (p), item discrimination (d), and item validity coefficients were calculated. For p, items were considered endorsed if participants selected one of the two highest options (e.g., “upset” or “very upset”). Because the OJS was positively skewed, low item endorsement was also calculated to determine which questions had proportionality higher instances of participants selecting one of the two lowest responses (e.g., “a little bit” or “not at all”). Items with extreme p values (10 items) and items that did not discriminate between high scorers and low scorers (2 items) were eliminated. Finally, item validity was assessed by correlating each OJS item with the relevant MJS subscale, which is important for adequate criterion validity (Cohen and Swerdlik, 2018). Eight items with the strongest item validity coefficients from each subscale were retained, for a final total of 24 items (see Appendix).
Reliability
Interitem consistency was analyzed using Cronbach’s alpha. The OJS scale had high reliability (α = 0.95) as did the individual subscales: emotional (α = 0.93), cognitive (α = 0.93), and behavioral (α = 0.90). Test–retest reliability was analyzed using Pearson-product moment correlations; the correlations between Time 1 and Time 2 scores were significant for the full scale (0.84, p < 0.001) and the emotional (0.76, p < 0.001) cognitive (0.74, p < 001) and behavioral (0.79, p < 0.001) subscales.
Construct validity
As expected, the correlations between the OJS and the Multidimensional Jealousy Scale (MJS) were high (Table 2) as were the correlations between corresponding subscales (e.g., OJS emotional and MJS emotional), which in turn were higher than the correlations between diverse subscales (e.g., OJS emotional and MJS cognitive). As expected, women (M = 2.97, SD = 0.90) scored significantly higher than men (M = 2.40, SD = 0.81) on the OJS (t = 3.26, p ≤ 0.001) and on the emotional (t = 2.78, p ≤ 0.01), cognitive (t = 2.75, p ≤ 0.05), and behavioral subscales (t = 2.79, p ≤ 0.01).
Table 2. Means, standard deviations and correlations for the online jealousy scale and multidimensional jealousy scale and their subscales, Study 1.
Convergent and discriminant validity
All measures were reliable and correlations followed the expected pattern (Table 3, first section). Interpersonal electronic surveillance and attachment anxiety were most strongly correlated with the OJS, social comparison and narcissism were moderately correlated, and relationship satisfaction and investment were weakly or not significantly correlated with the OJS. Regarding the OJS subscales, as expected, electronic surveillance was more strongly related to the behavioral subscales than the other two subscales; attachment anxiety and social comparison were more strongly correlated with the cognitive subscale than the other two subscales, and relationship satisfaction and investment were only significantly correlated with the cognitive subscale.
Table 3. Correlations between the online jealousy scale and validity scales.
Discussion
Study 1 provided strong evidence that the initial 24-item OJS was reliable, internally and over time. The pattern of associations with related constructs and factor analysis of the subscales provided evidence that the new scale measures what it intended to measure. One important limitation of this study was its homogeneous sample (dating college students ages 18–22). This calls into question the generalizability of the findings, especially given generational differences in the use of online communication. To determine whether the findings can be generalized to people of different ages and relationship types (e.g., married), a second study was conducted.
Study 2
The purpose of Study 2 was to (1) assess the scale’s reliability and validity in a larger, more heterogeneous sample and (2) further refine the measure. Examining generalizability is important because social media is now widely used by people of all ages (Pew Research Center, 2024). Studies using samples with a wide range of ages, although rare, indicate that associations between gender and online jealousy diminish with age and length of relationship (Saslow et al., 2013), perhaps because older individuals and individuals in longer-term relationships use social media for different purposes (Rus and Tiemensma, 2017). Given this and the evidence that jealousy decreases as people age (Demirtaş-Madran, 2018; Lantagne and Furman, 2017), significant negative correlations with age in a heterogeneous sample would provide further evidence of the scale’s validity.
Method
Participants in intimate relationships were recruited via Qualtrics Panels (n = 200). See Table 1 for demographic information. The measures and procedures were the same as those used in Study 1. The means and standard deviations for all measures can be seen in the second section of Table 3. Unlike Study 1, participants received $6.00 for completing Time 1 and another $6.00 for completing Time 2. Sixty-six percent of participants (n = 132) completed the Time 2 survey.
Because of the expected age variance, participants were asked whether they had experienced the types of hypothetical negative situations used in the OJS. Analyses for Study 2 were conducted on the full sample and on a subsample (n = 163) that excluded participants who had no experience with those scenarios. The pattern of results was the same; the findings for the full sample are reported here.
Results and discussion
Reliability
As we saw in Study 1, internal and test–retest reliability were strong. Cronbach’s alpha analyses indicated that the OJS had high internal consistency (α = 0.95) as did the individual subscales: emotional (α = 0.93), cognitive (α = 0.93), and behavioral (α = 0.90). The test–retest reliability coefficients were significant for the full OJS scale (0.82, p < 0.001) and the emotional (0.68, p < 0.001) cognitive (0.81, p < 001) and behavioral (0.94, p < 0.001) subscales.
Construct validity
As in Study 1, the correlations between the OJS and the MJS were strong, and the correlations between the same subscale (e.g., OJS emotional and MJS emotional) were higher than correlations between diverse subscales (e.g., OJS emotional and MJS cognitive and behavioral subscales) (Table 4).
Table 4. Means, standard deviations and correlations for the online jealousy scale and the multidimensional jealousy scale and their subscales, Study 2.
Demographic variables
No gender differences were found for the OJS emotional subscale (t = −0.87, ns); however, men reported higher levels of cognitive (t = 2.53, p ≤ 0.05) and behavioral (t = 2.25, p ≤ 0.001) jealousy. As expected, age was negatively correlated with the OJS (r = −0.44, p ≤ 0.001) and the emotional (r = −0.20, p ≤ 0.05), cognitive (r = −0.42, p ≤ 0.001), and behavioral (r = −0.48, p ≤ 0.001) subscales.
Convergent and discriminant validity
The means, standard deviations, and alpha coefficients for the validity scales can be seen in the second section of Table 3, along with the correlations with the OJS. All measures were reliable and, as expected, the OJS was strongly correlated with interpersonal surveillance and attachment anxiety. Furthermore, the correlation between interpersonal surveillance and the OJS was strongest with the behavioral subscale, and the correlations between attachment anxiety were strongest with the cognitive and behavioral subscales. Contrary to expectations, attachment anxiety had a higher correlation with behavioral jealousy than with cognitive jealousy, although Fisher’s r-to-z transformation indicated that the correlations were not significantly different.
Social comparison was moderately correlated with the OJS and the cognitive and behavioral subscales. Narcissism followed a similar pattern, although the correlation coefficients were higher. Relationship satisfaction was weakly correlated with the OJS and the emotional and cognitive subscales. Relational investment was weakly correlated with the OJS and the cognitive and behavioral subscales.
Taken together, these findings evidence the reliability, validity, and generalizability of the OJS. However, the cognitive and behavioral subscales were highly correlated and not always consistent with expectations, calling the use of the subscales into question. Item analysis was conducted to refine the measure further.
Item analysis
The process used to identify items for retention was the same as the process used in Study 1. Results confirmed that all the items loaded strongly on the expected factor (Supplementary Table S5). The emotional factor accounted for 62.17% of the variance, the cognitive factor accounted for an additional 16.78% and the behavioral factor accounted for an additional 3.28% of the variance, for a total of 82.22% of the variance accounted for. In contrast to Study 1, however, seven of the eight variables that loaded on the behavioral factor also had factor loadings above 0.50 on the cognitive subscale. Post hoc analyses of the factor structure of the MJS revealed the same results, indicating that the OJS factor loadings were more likely a reflection of the sample rather than idiosyncratic findings specific to the OJS measure. There were no extreme values for p or d, so items were retained based on validity coefficients (see Supplementary Material; items in bold were retained for the final, 12-item version). A third study was conducted to evaluate the final 12-item version.
Study 3
The primary purpose of Study 3 was to evaluate the factor validity of the 12-item version of the OJS using confirmatory factor analysis with another community sample. The secondary purpose was to test whether item consistency and construct and criterion validity were replicated in a second community sample.
Method
Participants in intimate relationships were recruited via Mturk (N = 1432). See Table 1 for demographic information. The measures and procedures were the same as those used in Study 2. The means and standard deviations for all measures can be seen in the third section of Table 3.
Results and discussion
To assess the factor validity of the scale, a confirmatory factor analysis was conducted using Amos Version 26 (Arbuckle, 2019). Good model fit is indicated by low chi-square values with probability values greater than 0.05, a root mean square error of approximation (RMSEA) below 0.08, and a comparative fit index (CFI) of 0.90 or greater (MacCallum et al., 1996). The analysis revealed that the model was a poor fit (χ2 = 189.64, p < 0.001; RMSEA = 0.14; CFI = 0.92). Modification indices indicated shared error variance among three pairs of endogenous variables. A second analysis was run that allowed those pairs to covary; the resulting model fit the data well (χ2 = 58.16, p = 0.15; RMSEA = 0.039; CFI = 0.99; see Supplementary Figure S1).
Correlations between the OJS and the validity scales can be seen in the third section of Table 3. The pattern of results was consistent with the findings in Study 1 and Study 2, providing further evidence of the construct validity of the OJS.
General discussion
Summary of findings
The overall purpose of these three studies was to develop, refine, and evaluate a measure of online jealousy. To develop the measure, we created a pool of items, modifying 25 items from the FBJ (Muise et al., 2009) and creating an additional 29 items meant to reflect the multidimensional nature of jealousy and online interactions more generally. Item analysis was used to identify optimal items in an initial sample of 111 undergraduates. In Study 2, the reliability and validity of the subsequent 24-item draft were assessed using a more heterogeneous sample. Evidence of the reliability and validity of the OJS was compelling. The scale was strongly related to another, well-established jealousy scale (MJS; Pfeiffer and Wong, 1989) and moderately correlated with the theoretically related constructs of interpersonal electronic surveillance and attachment anxiety. The OJS was weakly correlated with or unrelated to measures of social comparison, narcissism, relationship satisfaction, and relationship investment, consistent with past findings that these constructs were not strongly related to online jealousy, providing evidence of discriminant validity.
The pattern of associations among the OJS subscales (emotional, cognitive, and behavioral) was consistent with theory and past findings as well, providing further evidence of convergent and discriminant validity. The OJS subscales were more strongly correlated with the corresponding MJS subscale than with differing subscales. Interpersonal electronic surveillance was most strongly associated with the OJS behavioral subscale in Study 1 and behavioral and cognitive subscales in Study 2. Attachment anxiety was most strongly related to the cognitive subscale in Study 1 and the cognitive and behavioral subscales in Study 2.
Exploratory factor analyses in Study 1 and Study 2 suggested a three-factor solution, consistent with the multidimensional model of jealousy. Confirmatory factor analysis in Study 3 provided further evidence of the factor validity of the OJS.
The OJS and the Digital Jealousy Scale
As mentioned previously, a similar measure, the Digital Jealousy Scale (DJS; Gubler et al., 2023), was published as we were revising our manuscript. The study has notable strengths, including the use of large, international samples (participants were from Germany and the U.K.) and the succinct, unidimensional nature of the scale. The DJS will be useful when a shorter measure is desired and when making distinctions between emotional, cognitive and behavioral jealousy is not paramount. Since the DJS primarily assesses emotional jealousy (five items), with three items that assess behavioral jealousy and only one item that assesses cognitive jealousy, the multidimensional nature of the OJS makes it optimal for use in studies examining the distinct facets of jealousy. Further, the OJS also has a brief version, which includes only three more items than the DJS.
Limitations
There are methodological limitations that should be considered when interpreting these findings. The validity analyses were based on cross-sectional data, and thus only provided evidence of concurrent validity. Longitudinal studies are needed to examine predictive validity. All analyses were based on self-report data and reports of jealousy, attachment anxiety, and surveillance activities were subject to social desirability effects. Individuals rather than couples were used; although this is not unusual in the jealousy literature, dyadic analysis would allow for the examination of partner data as well as individual data. Further, the percentage of participants who identified as BIPOC varied across the samples: 56.7%, 14%, and 23.1% in Samples 1, 2, and 3, respectively, limiting the generalizability across ethnic groups. Finally, while we moved toward more inclusive language by using “preferred sex” instead of “opposite sex,” further development of inclusive language is needed to ensure that relationship assessments reflect the variance in sexual orientation (e.g., bisexual individuals), gender identity (e.g., non-binary individuals) and relationship type (e.g., polyamorous relationships). Finally, the age differences in the three samples were notable; especially between the first sample, with an average age of 19, and Samples 2 and 3, with averages of 51 and 41 years old, respectively. Given the evidence that jealousy diminishes with age and that older adults were exposed to technology later in life, it is likely that older individuals in relationships have less personal experience with online jealousy. Nevertheless, the finding that the OJS was highly reliable and valid in a college sample and an older, community-based sample, increases confidence of the usefulness of the measure.
Conclusion
As with any emerging area of study, the burgeoning subfield of communication technology and intimate relationships requires the basic building blocks of valid and reliable measurements to flourish. Adapting measures created for offline situations can be useful, but psychometric analysis is required to ensure that adapted or newly developed scales are consistently and accurately measuring the constructs of interest in the digital environment. In the absence of such measures, researchers must be wary about drawing conclusions across studies.
A valid and reliable instrument for measuring online jealousy is particularly needed as jealousy has emerged as a key variable in the study of online communications and social media in relationships, and the measurement of online jealousy has varied widely across studies (Rus and Tiemensma, 2017), limiting our ability to build on previous findings. The OJS, adapted in part from the Facebook Jealousy Scale (Muise et al., 2009), was created to meet that need. The findings across both studies indicate that the OJS can be reliably used to measure jealousy related to social media for research participants of all ages and in various types of relationships.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving humans were approved by Santa Clara University Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.
Author contributions
KS: Writing – original draft, Writing – review & editing. KB: Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that no financial support was received for the research, authorship, and/or publication of this article.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fhumd.2024.1447003/full#supplementary-material
Footnotes
1. ^As we were completing a revision of this article, a study describing a new measure, the Digital Jealousy Scale (DJS), was published (Gubler et al., 2023). The DJS differs substantially from our measure (e.g., it is much briefer and assesses jealousy as a single factor). Differences between the two measures are addressed in detail in the General Discussion section.
2. ^Boateng and collogues suggest at least 10 participants per item for confirmatory analyses. Thus, a sample size of 143 is adequate for this 12-item measure (Boateng et al., 2018).
References
Appel, H., Gerlach, A. L., and Crusius, J. (2016). The interplay between Facebook use, social comparison, envy, and depression. Curr. Opin. Psychol. 9, 44–49. doi: 10.1016/j.copsyc.2015.10.006
Auxier, B, and Anderson, M. (2021). Social media use in 2021 [internet]. Pew Research Center: Internet, Science & Tech. Available at: https://www.pewresearch.org/internet/2021/04/07/social-media-use-in-2021/ (Accessed August 29, 2022)
Bevan, J. L. (2008). Experiencing and communicating romantic jealousy: questioning the investment model. South Commun. J. 73, 42–67. doi: 10.1080/10417940701815626
Bevan, J. L. (2017). Romantic jealousy in face-to-face and technologically-mediated interactions: a communicative interdependence perspective. West. J. Commun. 81, 466–482. doi: 10.1080/10570314.2017.1283048
Bevan, J. L. (2018). Social networking site password sharing and account monitoring as online surveillance. Cyberpsychol. Behav. Soc. Netw. 21, 797–802. doi: 10.1089/cyber.2018.0359
Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., and Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health 6:149. doi: 10.3389/fpubh.2018.00149
Buffardi, L. E., and Campbell, W. K. (2008). Narcissism and social networking web sites. Personal. Soc. Psychol. Bull. 34, 1303–1314. doi: 10.1177/0146167208320061
Cohen, R. J., and Swerdlik, M. E. (2018). Psychological testing and assessment: An introduction to tests and measurement. 9th Edn. New York, NY: McGraw-Hill.
Dandurand, C., and Lafontaine, M. (2014). Jealousy and couple satisfaction: a romantic attachment perspective. Marriage Family Rev. 50, 154–173. doi: 10.1080/01494929.2013.879549
Del Sánchez-Fuentes, M., Moyano, N., Alcaraz-Iborra, M., Parra-Barrera, S. M., and Quílez-Robres, A. (2023). Validación de la Escala Abreviada de Celos de Facebook entre Hombres y Mujeres Colombianos. Acta Colombiana Psicol. 26, 50–64. doi: 10.14718/ACP.2023.26.2.5
Demirtaş-Madran, H. A. (2018). Relationship among Facebook jealousy, aggression, and personal and relationship variables. Behav. Inform. Technol. 37, 462–472. doi: 10.1080/0144929X.2018.1451919
DiBello, A. M., Rodriguez, L. M., Hadden, B. W. W., and Neighbors, C. (2015). The green eyed monster in the bottle: relationship contingent self-esteem, romantic jealousy, and alcohol-related problems. Addict. Behav. 49, 52–58. doi: 10.1016/j.addbeh.2015.05.008
Drouin, M., Miller, D. A., and Dibble, J. L. (2014). Ignore your partners’ current Facebook friends; beware the ones they add! Comput. Hum. Behav. 35, 483–488. doi: 10.1016/j.chb.2014.02.032
Elphinston, R. A., and Noller, P. (2011). Time to face it! Facebook intrusion and the implications for romantic jealousy and relationship satisfaction. Cyberpsychol. Behav. Soc. Netw. 14, 631–635. doi: 10.1089/cyber.2010.0318
Foster, J. D., and Campbell, W. K. (2005). Narcissism and resistance to doubts about romantic partners. J. Res. Pers. 39, 550–557. doi: 10.1016/j.jrp.2004.11.001
Fox, J., and Warber, K. M. (2014). Social networking sites in romantic relationships: attachment, uncertainty, and partner surveillance on Facebook. Cyberpsychol. Behav. Soc. Netw. 17, 3–7. doi: 10.1089/cyber.2012.0667
Frampton, J. R. (2024). An examination of specialness meaning framework threat in reactive and retroactive romantic jealousy experiences. Pers. Relat. 31, 190–211. doi: 10.1111/pere.12526
Funk, J. L., and Rogge, R. D. (2007). Testing the ruler with item response theory: increasing precision of measurement for relationship satisfaction with the couples satisfaction index. J. Family. Psychol. 21, 572–583. doi: 10.1037/0893-3200.21.4.572
Gibbons, F. X., and Buunk, B. P. (1999). Individual differences in social comparison: Development of a scale of social comparison orientation. J. Pers. Soc. Psychol. 76, 129–142. doi: 10.1037/0022-3514.76.1.129
Gubler, D. A., Schlegel, K., Richter, M., Kapanci, T., and Troche, S. J. (2023). The green-eyed monster in social media – development and validation of a digital jealousy scale. Psychol. Test Adapt. Develop. 4, 13–27. doi: 10.1027/2698-1866/a000033
Imperato, C., Everri, M., and Mancini, T. (2023). Does Facebook ‘threaten’ romantic relationships? Online surveillance and couple visibility behaviours in romantic jealousy and couple relationship quality in a sample of Italian women. J. Fam. Stud. 29, 792–806. doi: 10.1080/13229400.2021.1987295
Kelley, H. H., and Thibaut, J. W. (1978). Interpersonal relations: A theory of interdependence. New York: Wiley.
Lantagne, A., and Furman, W. (2017). Romantic relationship development: the interplay between age and relationship length. Dev. Psychol. 53, 1738–1749. doi: 10.1037/dev0000363
MacCallum, R. C., Browne, M. W., and Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychol. Methods 1, 130–149. doi: 10.1037/1082-989X.1.2.130
Marshall, T. C., Bejanyan, K., di Castro, G. A. I. A., and Lee, R. A. (2013). Attachment styles as predictors of Facebook-related jealousy and surveillance in romantic relationships. Pers. Relat. 20, 1–22. doi: 10.1111/j.1475-6811.2011.01393.x
McAndrew, F. T., and Shah, S. S. (2013). Sex differences in jealousy over Facebook activity. Comput. Hum. Behav. 29, 2603–2606. doi: 10.1016/j.chb.2013.06.030
McCain, J. L., and Campbell, W. K. (2018). Narcissism and social media use: a meta-analytic review. Psychol. Pop. Media Cult. 7, 308–327. doi: 10.1037/ppm0000137
Meier, A., and Schafer, S. (2018). The positive side of social comparison on social network sites: how envy can drive inspiration on Instagram. Cyberpsychol. Behav. 21, 411–417. doi: 10.1089/cyber.2017.0708
Miller, M. J., Denes, A., Diaz, B., and Buck, R. (2014). Attachment style predicts jealous reactions to viewing touch between a romantic partner and close friend: implications for internet social communication. J. Nonverbal Behav. 38, 451–476. doi: 10.1007/s10919-014-0196-y
Muise, A., Christofides, E., and Desmarais, S. (2009). More information than you ever wanted: does Facebook bring out the green-eyed monster of jealousy? Cyberpsychol. Behav. 12, 441–444. doi: 10.1089/cpb.2008.0263
Muise, A., Christofides, E., and Desmarais, S. (2014). “Creeping” or just information seeking? Gender differences in partner monitoring in response to jealousy on Facebook. Pers. Relat. 21, 35–50. doi: 10.1111/pere.12014
Pew Research Center (2024). Social media fact sheet. Available at: https://www.pewresearch.org/internet/fact-sheet/social-media/ (Accessed October 1, 2024)
Pfeiffer, S., and Wong, P. (1989). Multidimensional jealousy. J. Soc. Pers. Relat. 6, 181–196. doi: 10.1177/026540758900600203
Pincus, A. L. (2013). “The pathological narcissism inventory” in Understanding and treating pathological narcissism. ed. J. S. Ogrodniczuk (American Psychological Association), 93–110.
Quiroz, S. I., Ha, T., and Anderson, S. F. (2024). “You liked that Instagram post?!” adolescents’ jealousy and digital dating abuse behaviors in reaction to digital romantic relationship threats. Comput. Hum. Behav. 153:108111. doi: 10.1016/j.chb.2023.108111
Rodriguez, L. M., DiBello, A. M., Øverup, C. S., and Neighbors, C. (2015). The price of distrust: trust, anxious attachment, jealousy, and partner abuse. Partn. Abus. 6, 298–319. doi: 10.1891/1946-6560.6.3.298
Rus, H., and Tiemensma, J. (2017). “It's complicated.” a systematic review of associations between social network site use and romantic relationships. Comput. Hum. Behav. 75, 684–703. doi: 10.1016/j.chb.2017.06.004
Rusbult, C. E., Martz, J. M., and Agnew, C. R. (1998). The investment model scale: measuring commitment level, satisfaction level, quality of alternatives, and investment size. Pers. Relat. 5, 357–387. doi: 10.1111/j.1475-6811.1998.tb00177.x
Rydell, R. J., and Bringle, R. G. (2007). Differentiating reactive and suspicious jealousy. Soc. Behav. Personal. Int. J. 35, 1099–1114. doi: 10.2224/sbp.2007.35.8.1099
Salovey, P., and Rodin, J. (1986). The differentiation of social-comparison jealousy and romantic jealousy. J. Pers. Soc. Psychol. 50, 1100–1112. doi: 10.1037/0022-3514.50.6.1100
Saslow, L. R., Muise, A., Impett, E. A., and Dubin, M. (2013). Can you see how happy we are? Facebook images and relationship satisfaction. Soc. Psychol. Personal. Sci. 4, 411–418. doi: 10.1177/1948550612460059
Smith, R., Alicke, M. D., and Zell, E. (2008). “Social comparison and envy” in Envy. ed. R. H. Smith (Oxford University Press), 73–93.
Social Media Fact Sheet. (2021). Pew research center: internet, Science & Tech. Available at: https://www.pewresearch.org/internet/fact-sheet/social-media (Accessed August 29, 2022)
Tandoc, E. C. Jr., and Goh, Z. H. (2023). Is Facebooking really depressing? Revisiting the relationships among social media use, envy, and depression. Inf. Commun. Soc. 26, 551–567. doi: 10.1080/1369118X.2021.1954975
Telli, A., and Yavuz Güler, Ç. (2023). Differentiation of self, forgiveness, jealousy, and conflict resolution responses among married individuals: the mediating role of relationship satisfaction, relationship adjustment, and emotional dependency. Contemp. Fam. Ther. 45, 157–171. doi: 10.1007/s10591-021-09603-8
Tokunaga, R. S. (2011). Friend me or you’ll strain us: Understanding negative events that occur over social networking sites. Cyberpsychology, Behavior, and Social Networking. 14, 425–432. doi: 10.1089/cyber.2010.0140
Tokunaga, R. S. (2016). Interpersonal surveillance over social network sites. J. Soc. Pers. Relat. 33, 171–190. doi: 10.1177/0265407514568749
Tortoriello, G. K., Hart, W., Richardson, K., and Tullett, A. M. (2017). Do narcissists try to make romantic partners jealous on purpose? An examination of motives for deliberate jealousy-induction among subtypes of narcissism. Personal. Individ. Differ. 114, 10–15. doi: 10.1016/j.paid.2017.03.052
Utz, S., and Beukeboom, C. J. (2011). The role of social network sites in romantic relationships: effects on jealousy and relationship happiness. J. Comput.-Mediat. Commun. 16, 511–527. doi: 10.1111/j.1083-6101.2011.01552.x
Vogels, EA, Gelles-Watnick, R, and Massarat, N. (2022). Teens, social media and technology 2022 [internet]. Pew Research Center: Internet, Science & Tech. Available at: https://www.pewresearch.org/internet/2022/08/10/teens-social-media-and-technology-2022/ (Accessed August 29, 2022)
Wang, X., Wang, S., Wang, H., and Dong, W. (2024). Females suffer more from partner phubbing? The roles of romantic jealousy and relational aggression between partner phubbing and intimacy quality. Curr. Psychol. 43, 20250–20262. doi: 10.1007/s12144-024-05826-8
Wegner, R., Roy, A. R. K., Gorman, K. R., and Ferguson, K. (2018). Attachment, relationship communication style and the use of jealousy induction techniques in romantic relationships. Personal. Individ. Differ. 129, 6–11. doi: 10.1016/j.paid.2018.02.033
Keywords: social media, romantic jealousy, psychometrics, intimate relationships, dating relationships
Citation: Sullivan KT and Bruchmann K (2025) The Online Jealousy Scale: an adaptation, extension, and psychometric analysis of the Facebook Jealousy Scale. Front. Hum. Dyn. 6:1447003. doi: 10.3389/fhumd.2024.1447003
Edited by:
Pragya Lodha, Lokmanya Tilak Municipal General Hospital, IndiaReviewed by:
Sascha Schwarz, University of Wuppertal, GermanyDesmond Onyemechi Okocha, ECWA Bingham University, Nigeria
Copyright © 2025 Sullivan and Bruchmann. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Kieran T. Sullivan, a3N1bGxpdmFuQHNjdS5lZHU=