Psychometric Properties of the Nine-Item Problematic Internet Use Questionnaire (PIUQ-9) in a Lithuanian Sample of Students

Objectives: To date, there is no reliable instrument which could be used to assess problematic Internet use (PIU) in Lithuania. The nine-item Problematic Internet Use Questionnaire (PIUQ-9) previously validated in multiple countries, could be a potential tool for measuring PIU severity. The main objective of the present study was to explore the psychometric properties of the Lithuanian version of the questionnaire.

Methods: A total of 272 students (17% men, mean age 27 ± 9 years) completed the PIUQ-9, the Patient Health Questionnaire (PHQ) and answered questions about the impairment of daily functioning caused by PIU in an online survey.

Results: A confirmatory factor analysis indicated that a bi-factor model with one general factor “general problem” and two-specific factors “obsession” and “neglect + control disorder” fitted the data well. The presence of a strong global factor was supported by the common variance index in the bi-factor model indicating that the “general problem” factor explained 67.7% of common variance. The multiple indicators multiple causes (MIMIC) model showed that psychiatric symptoms (β = 0.25) had a moderate, while impairment due to PIU (β = 0.41) had a moderate-to-strong direct effect on the factor “general problem” supporting the construct validity of the scale.

Conclusion: The Lithuanian version of the PIUQ-9 has appropriate psychometric properties to be used in measuring PIU severity in student samples.

Introduction

There is a growing global concern about the public health problems and societal costs of problematic Internet use (PIU), which has an estimated prevalence of 1–27% within the general population (1). PIU is increasingly recognized as disproportionately impacting young people and represents an emerging mental health research challenge (2). Excessive social media use, online streaming, gaming, gambling, online shopping, email checking and pornography viewing are all examples of behaviors that have the potential to cause significant impairment of everyday functioning. Various factors may contribute to PIU including age, gender, socioeconomic status as well as symptoms of mental health conditions or personality traits leading to even more advanced engagement in problematic online behaviors. Furthermore, there is a huge risk that any of these behaviors could have the tendency to increase during and after the COVID-19 quarantine period (3) and may require professional intervention and support. However, it is crucial to have adequate adapted measurements for PIU before developing appropriate tailored interventions.

National health authorities are now expressing concern over the potential impact of PIU. However, there remains both a lack of agreement on what constitutes PIU and scientific data on its prevalence, clinical parameters, and socio-economic burden (2). In order to start answering these important scientific questions, we need an operational definition and reliable tools to measure PIU severity.

While the National Department of Statistics in Lithuania reports that in 2019 ~98% of people aged 16–24 and 95% of people aged 25–34 had been using the Internet daily, there are only few studies investigating PIU prevalence. However, most of the existing studies in European countries which include Lithuania have focused on problematic online behaviors in children and teenagers (4, 5). In a cross-sectional study of 1806 13–18 years old schoolchildren in Lithuania, 9.1% of the study subjects showed markedly expressed symptoms of internet addiction as measured with the Young Diagnostic Questionnaire (6). This particular study also found that symptoms of Internet addiction were also more prevalent among male than female teenagers. Nevertheless, no studies have investigated PIU in young adults and only two reliable questionnaires were used to assess symptoms of Internet addiction in teenagers (4, 5). Therefore, it is crucial to investigate this particular study group of young adults and identify a questionnaire which could be suitable for a specific assessment. A recent publication by Laconi et al. (7) investigated the psychometric properties of the Nine-Item Problematic Internet Use Questionnaire (PIUQ-9) across various different language-based samples of European internet users, highlighting that a brief scale consisting of only nine questions could reliably screen for PIU. The scale has not been used in Lithuania before and its psychometric properties, validated in other cultural backgrounds, show promising results.

Our report attempts to validate a Lithuanian language version of the PIUQ-9 scale. To the best of our knowledge, Lithuania does not a have an adapted measure for PIU severity. Given the brevity of the scale, PIUQ-9 would be useful for researchers, clinicians and educators interested in measuring PIU severity.

Method

Procedure

Participants completed an online survey, which provided information about the study (i.e., the study objectives and assurances of anonymity and confidentiality). An online survey was made available from 2019 September until 2019 November using Google Forms. A snowballing technique was used to develop convenience sample targeting students. Firsthand networks of study volunteers and student council assistants were employed by researchers to spread the link. The initial link was shared to Lithuanian University of Health Sciences students using social network services and messenger services such as Facebook or Skype. Researchers encouraged participants to send the link to other students, helping to reach an adequate study sample. However, the engagement rate was not monitored. A formal online consent was provided for each participant to tick before starting the survey. No incentives were given upon completion.

The study received approval from the Bioethics committee and conformed to the principles outlined in the Declaration of Helsinki. The survey included sociodemographic information, the PIUQ-9 questionnaire, and several subscales of the Patient Health Questionnaire (PHQ).

Measures

Besides collecting basic information on age and gender, we also assessed impairment due to PIU by asking study participants a single question with yes/no answer on whether they thought that internet usage impacted their ability to perform daily activities and/or engage in desired social interactions.

The nine-item version of the Problematic Internet Use Questionnaire (PIUQ-9) was used to assess PIU (8) (Refer to the Lithuanian language version in Appendix A). The PIUQ-9 is a short version of the 18-item Problematic Internet Use Questionnaire [PIUQ; (9)]. Originally, both scales were reported to have three subscales: obsession, neglect, and control disorder. Based on the previous study examining the psychometric properties of the PIUQ-9, the short scale has a bi-factor structure including one “general problem” factor and two-specific factors of “obsession” and “neglect + control disorder.” The severity of PIU is assessed through a five point Likert scale, ranging from “never” to “always/almost always.” The sum score of the questionnaire ranges from nine to 45. A Higher score indicates a higher risk of PIU. The internal consistency of the PIUQ-9 was 0.89 in this particular study sample. As in previous studies (7), all survey questions were translated from English to Lithuanian using the procedure of double back-translation. An experienced psychiatrist (VS) conducted the translation and the back translation was performed by a clinical psychologist (JB). The final consensus was reached by discussing significant differences in the back translation as compared to the original English version.

Psychiatric disorder symptoms were assessed using the Patient Health Questionnaire (PHQ) (10). The PHQ consists of several modules assessing symptoms of several psychiatric disorders, including depressive disorder, anxiety disorders, somatoform disorder and alcohol abuse/dependence. The depression module consisted of nine items questioning depression severity over a 2 weeks period with possible scores ranging from zero (“not at all”) to three (“nearly everyday”). The sum score of depression symptomatology ranged from zero to 27. The anxiety module contained seven questions regarding the severity of anxiety symptom levels during a 4 weeks period. Each question was rated on a scale ranging from zero (“not at all”) to two (“more than half the days”) and resulting in a total score ranging from zero to 14. The somatoform module consisted of 13 items regarding physical problems that bothered individuals over a 4 weeks period. Each item ranged from zero (“not bothered”) to two (“bothered a lot”) resulting in a total score ranging from zero to 26. The alcohol abuse/dependence module contained five questions with answers “yes” and “no” asking about the presence of problematic behaviors related to alcohol use occurring more than once in the past 6 months. The total score for the module ranged from zero to five. For the statistical modeling, a general score for the questionnaire summing up the four aforementioned modules was used in order to evaluate the global severity of any psychiatric symptomatology. The internal consistency of PHQ in the current sample was 0.91.

Statistical Analysis

The SPSS for Windows, version 17.0 (SPSS Inc., Chicago, Illinois) and Mplus 8 (11) were employed for statistical procedures. All statistical procedures were based on the Laconi et al. study validating the psychometric properties of the PIUQ-9 (7). Using Student's t-tests we have also compared the PIUQ-9 scores and impairment to engage in daily activities and social life due to excessive internet use between genders. In brief, four alternative factor structures were tested: (i) the three-factor model proposed in the study by Koronczai et al. (8), (ii) a two-factor model where control disorder and neglect factors were merged into one due to the high correlation, (iii) a bi-factor model comprising a dimension of global severity on which each item is loaded, plus three specific factors on which the items belonging to the obsession, neglect and control disorder were loaded and where the correlations between specific factors were fixed at zero; (iii) a bi-factor model representing a global severity dimension plus two specific factors (obsession and neglect + control disorder). The correlations between specific factors were also fixed at zero. The degree of fit of the factor models were measured through confirmatory factor analysis (CFA) with maximum likelihood estimation with robust standard errors (MLR).

The following fit indices were used to evaluate model fit: the ratio of the chi-square to its degrees of freedom (χ²/df ratio), the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the goodness of fit index (GFI), the root mean square error of approximation (RMSEA) and its p close value, and the standardized root mean residual (SRMR). The following values were required to indicate adequate fits: χ²/df ratio ≤ 5 and preferably ≤ 2; CFI ≥ 0.95; TLI ≥ 0.95; GFI ≥ 0.9, RMSEA < 0.08, SRMR ≤ 0.10 (12).

The explained common variance (ECV) index was used to measure the degree of unidimensionality in the bi-factor model and the percentage of general factor common variance. Omega and omega hierarchical indices were chosen to measure the ability of the PIUQ-9 scores to estimate the combination of factorial constructs, as well as specific target constructs. The index of construct replicability or H index was used for the evaluation of specific factors (≥0.70 was considered as high).

CFA with covariates or multiple indicators multiple causes (MIMIC) confirmatory factor analysis was performed in the Lithuanian language sample to test the construct validity of the best fitting structure (i.e., the bi-factor-model with two specific factors) by exploring correlations between the general and specific factors and four probable predictors (i.e., age, gender, psychiatric symptoms severity, and impairment due to PIU).

Results

Descriptive Statistics

This study consisted of 291 students studying at the Lithuanian University of Health Sciences. Nineteen participants (7%) refused to participate in the study, leaving the total study sample of 272 participants (83% women and 17% men, mean age 27 ± 9 years). Thirty-nine participants (14%) indicated that PIU caused significant impairment in their daily living and/or social activities. Neither the PIUQ-9 scores, nor impairment to engage in daily activities and social life due to excessive internet use differed between genders (p > 0.05).

The Model of the PIUQ-9

The X² test was significant in all four factor models. The bi-factor model comprising two specific factors appeared to be the best fitting model according to the fit indices CFI, TLI, RMSEA, and SRMR (Table 1).

TABLE 1

Table 1. Confirmatory factor analysis of four measurement models of PIUQ-9.

Factor loadings for the bi-factor model with two specific factors are presented in Table 2. All nine items had significant factor loadings on the global PIU factor. The ninth item for the specific factor “obsession” was found to be non-significant; however, this item had strong significant loadings on the main factor. This was the case for items five, seven, and eight in the specific factor “neglect + control,” as these items were not significant on the specific factor, despite having strong significant loadings on the general problem factor.

TABLE 2

Table 2. Standardized factor loadings of the bifactor model with two specific factors of PIUQ-9.

According to the ECV index of the bi-factor model, the general problem factor in the PIUQ-9 explained 67.7% of common variance, supporting a strong global factor. Table 3 shows ECV indices of specific factors. Evaluation of the precision of a questionnaire to assess the combination of general and specific factors, and a particular target construct was carried out through the calculation of the omega and omega hierarchical coefficients. For the Lithuanian sample, the specific factor “obsession” omega coefficient was 80.7 and 82.7% for the merged specific factor. The omega hierarchical coefficient for the “obsession” specific factor was 20.9% and in the case of the merged specific factor it was 12.8%. With the specific factors explaining ~13–21% of the PIU score these coefficients are considered significant. The H value above >0.70 (i.e., 0.89) indicated the “neglect + control disorder” to have good construct replicability. The obsession factor had a good replicability as well with the H value of 0.92.

TABLE 3

Table 3. Indicators of dimensionality and reliability of the bifactor model of PIUQ-9.

The MIMIC model shown in Table 4 indicates correlations between the general and specific factors and PIU predictors, such as age, gender, psychiatric symptoms, and impairment due to PIU. Generally, psychiatric symptoms had a moderate direct effect on the general problem factor (β = 0.26). Impairment caused due to PIU had a moderate-to-strong direct effect on the general problem factor (β = 0.41). The effect of psychiatric symptoms on the two specific factors, namely “obsession” and “neglect + control disorder” was also moderate (β = 0.21; β = 0.24, respectively). Furthermore, very similar effects of the impairment due to PIU were observed on the two specific factors (β = 0.39; β = 0.38, respectively).

TABLE 4

Table 4. Multiple indicators multiple causes (MIMIC) model with standardized coefficients (gender and age were both controlled for in the models).

Discussion

The main objective of the present study was to explore the psychometric characteristics of the PIUQ-9. Descriptive results available in our study correspond to the Laconi et al. data (7), for example the mean age of the German sample was 27 ± 10, the mean age of the total sample was 26 ± 9, while the mean age in our sample was 27 ± 9 years. The percentage of men involved in the study dramatically varied between countries in the aforementioned international study (7) ranging from 0.4 to 97% with a mean of 38%. Our study involved a relatively low number of male individuals comprising 17% of the sample. While many of the PIU studies are overrepresented by male subjects (2), our study might better reflect characteristics of Internet use in female students. However, PIUQ-9 scores as well as impairment related to the performance of daily and social activities due to excessive Internet use did not differ between genders.

The mean score of 18.4 ± 6.3 of PIUQ-9 in the Lithuanian sample was very close to the mean score of the Polish sample reported in Laconi et al.'s study (7). Lithuania is culturally closest to Poland, thus it is possible that PIU symptomatology severity corresponds. The percentage of impairment caused by PIU (14%) corresponds to the data on the European PIU prevalence in youth from 11 countries (13). However, the impairment percent in our study is slightly higher than in the Lithuanian study investigating schoolchildren (6). Several reasons might explain the discrepancy. First, students are the group with the most expressed Internet consumption. Secondly, problematic use and true clinical impairment due to PIU while overlapping are still two different concepts, which measured with different questionnaires might produce discrepancy. Thus, it is crucial to have a questionnaire which could reliably identify PIU symptomatology in prevalence studies.

Our results have confirmed good fit indices for the bi-factor model, with one “general problem” factor and two specific factors of “obsession” and “neglect + control disorder.” Psychiatric symptoms had a moderate direct effect on the general problem factor in the Lithuanian sample, showing good construct validity of the scale. Impairment related to the performance of daily and social activities due to excessive Internet use was a moderate-to-strong predictor of PIU.

The major limitation of the present study is the cross-sectional design, thereby precluding us from making causal assumptions. The Lithuanian sample consisted of students attending one university and was a comparatively small sample size. Furthermore, the gender distribution of the sample was skewed, with a considerably higher proportion of female students. Thus, the results may not represent general or clinical populations and may be subject to selection bias. Questions on educational level (14), and socioeconomic status were not included into our survey, limiting a broader scope of the PIU context in students. While we changed the time spent online covariate with the perceived impairment one, we consider that not including this criterion is a significant limitation of this study. Differentiating time spent online on weekend might have further contributed to the general factor of the questionnaire (7).

To the best of our knowledge this is the first study validating the PIUQ-9 questionnaire in the Lithuanian language. The Lithuanian version of the PIUQ-9 has demonstrated appropriate psychometric characteristics and is suitable for screening PIU in university students. Self-reported perceived impairment related to Internet usage is a strong predictor of general PIU psychopathology. Our current study adds PIUQ-9 as a validated tool for clinicians and educators to screen for the risk and severity of PIU. During a time with an increased Internet usage due to COVID-19 requirements to self-isolate or study remotely, PIUQ-9 could be a potential tool in measuring PIU severity. Future longitudinal studies should observe PIU changes over time and establish a cut-off threshold to define individuals who are at possible risk of PIU and therefore should be advised to seek treatment.

In summary, the PIUQ-9 has good psychometric characteristics and can be applied in research, as well as for educational and clinical purposes, to screen for the risk and severity of PIU.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Lithuanian University of Health Sciences Bioethics Center. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

JB, VS, ZD, and OK conceived and designed the study. VS and JB were responsible for data collection and evaluation. Statistical analyses were performed by AP. JB prepared the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This project has received funding from the Research Council of Lithuania (LMTLT), agreement no. S-GEV-20-5. OK was supported by the János Bolyai Research Scholarship of the Hungarian Academy of Sciences and by the ÚNKP-20-5 New National Excellence Program of the Ministry for Innovation and Technology from the source of the National Research, Development and Innovation Fund. ZD was supported by the Hungarian National Research, Development and Innovation Office (Grant Numbers: KKP126835 and NKFIH-1157-8/2019-DT).

Conflict of Interest

In the past several years, JB has been serving as a consultant for Cogstate, Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This publication is based upon work from COST Action CA16207 European Network for Problematic Usage of the Internet, supported by COST (European Cooperation in Science and Technology). www.cost.eu.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2020.565769/full#supplementary-material

References

1. Kuss JD, Griffiths DM, Karila L, Billieux J. Internet addiction: a systematic review of epidemiological research for the last decade. Curr Pharm Design. (2014) 20:4026–52. doi: 10.2174/13816128113199990617

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Fineberg NA, Demetrovics Z, Stein DJ, Ioannidis K, Potenza MN, Grünblatt E, et al. Manifesto for a European research network into problematic usage of the internet. Eur Neuropsychopharmacol. (2018) 28:1232–46. doi: 10.1016/j.euroneuro.2018.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Király O, Potenza MN, Stein DJ, King DL, Hodgins DC, Saunders JB, et al. Preventing problematic internet use during the COVID-19 pandemic: consensus guidance. Compr Psychiatry. (2020) 100:152180. doi: 10.1016/j.comppsych.2020.152180

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Blinka L, Škarupová K, Ševčíková A, Wölfling K, Müller KW, Dreier M. Excessive internet use in European adolescents: what determines differences in severity? Int J Public Health. (2015) 60:249–56. doi: 10.1007/s00038-014-0635-x

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Škarupová K, Ólafsson K, Blinka L. Excessive internet use and its association with negative experiences: quasi-validation of a short scale in 25 European countries. Comp Human Behav. (2015) 53:118–23. doi: 10.1016/j.chb.2015.06.047

CrossRef Full Text | Google Scholar

6. Ustinavičiene R, Škemiene L, Lukšiene D, Radišauskas R, Kaliniene G, Vasilavičius P. Problematic computer game use as expression of internet addiction and its association with self-rated health in the Lithuanian adolescent population. Medicina. (2016) 52:199–204. doi: 10.1016/j.medici.2016.04.002

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Laconi S, Urbán R, Kaliszewska-Czeremska K, Kuss DJ, Gnisci A, Sergi I, et al. Psychometric evaluation of the nine-item Problematic Internet Use Questionnaire (PIUQ-9) in nine European samples of internet users. Front Psychiatry. (2019) 10:136. doi: 10.3389/fpsyt.2019.00136

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Koronczai B, Urbán R, Kökönyei G, Paksi B, Papp K, Kun B, et al. Confirmation of the three-factor model of problematic internet use on off-line adolescent and adult samples. Cyberpsychol Behav Soc Netw. (2011) 14:657–64. doi: 10.1089/cyber.2010.0345

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Demetrovics Z, Szeredi B, Rózsa S. The three-factor model of internet addiction: the development of the Problematic Internet Use Questionnaire. Behav Res Methods. (2008) 40:563–74. doi: 10.3758/BRM.40.2.563

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Spitzer RL, Williams JB, Kroenke K, Hornyak R, McMurray J, Group PHQO-GS. Validity and utility of the PRIME-MD patient health questionnaire in assessment of 3000 obstetric-gynecologic patients: the PRIME-MD Patient Health Questionnaire Obstetrics-Gynecology Study. Am J Obstetr Gynecol. (2000) 183:759–69. doi: 10.1067/mob.2000.106580

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Muthén L, Muthén B. Mplus User's Guide. 8th ed. Los Angeles, CA: Muthén & Muthén (1998).

12. Brown TA. Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Publications (2015).

Google Scholar

13. Durkee T, Kaess M, Carli V, Parzer P, Wasserman C, Floderus B, et al. Prevalence of pathological internet use among adolescents in Europe: demographic and social factors. Addiction. (2012) 107:2210–22. doi: 10.1111/j.1360-0443.2012.03946.x

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Shepperd JA, Emanuel AS, Dodd VJ, Logan HL. The reliability of psychological instruments in community samples: a cautionary note. J Health Psychol. (2016) 21:2033–41. doi: 10.1177/1359105315569859

PubMed Abstract | CrossRef Full Text | Google Scholar