Shuzhen Niu
Qian Wu
Silian Ding2- 1Shanghai Tenth People's Hospital, School of Medicine, Tongji University, Shanghai, China
- 2Shanghai Fourth People's Hospital, Shanghai, China
- 3Department of Nephrology, Shanghai Zhabei Central Hospital, Shanghai, China
Objective: This study investigated the consistency and determined the optimal threshold values of three scales in the diagnosis of insomnia of ischemic stroke (IS) patients.
Methods: Participants in this study consisted of 569 acute IS patients. All 569 patients completed the assessment of the three insomnia scales. Insomnia of IS patients were assessed by Pittsburgh sleep quality index (PSQI), Insomnia Severity Index (ISI), and Athens insomnia scale (AIS). Also, basic patient information, neurological function, and activities of daily living were assessed. General information was compared between the insomnia group and the no-insomnia group. Cronbach’s α coefficients, Cohen’s Kappa consistency, Receiver operating characteristic (ROC) curve and DeLong’s test analysis were used to analyze the reliability and diagnostic validity of PSQI, ISI, and AIS.
Results: The PSQI and ISI showed high reliability with Cronbach’s α of 0.875 and 0.858, respectively, while the AIS had an α coefficient of 0.734, demonstrating acceptable reliability. The PSQI, ISI, and AIS showed outstanding diagnostic ability with an AUC of 0.960 (95% CI: 0.946, 0.974), 0.911 (95% CI: 0.882, 0.941), and 0.876 (95% CI:0.837, 0.916). The best diagnostic cutoffs for PSQI, ISI, and AIS are ≥9, ≥15, and ≥8.
Conclusion: Each of the three questionnaires has advantages and disadvantages when assessing insomnia. In the evaluation of insomnia in IS patients, the best questionnaire selection should be made according to the purpose of clinical evaluation and considering the sensitivity and specificity.
1. Introduction
Stroke is a leading cause of death and disability around the world (1). Ischemic stroke (IS) has high rates of morbidity, disability, recurrence and mortality (2). Insomnia is one of the common complaints of IS survivors. In contrast to other long-term sequelae of IS such as mobility and cognitive impairment, insomnia has received less attention and related research, despite being a risk factor for stroke (3). In accordance with the latest manual by the American Association of Sleep Medicine (AASM), insomnia is defined as persistent sleep problems and daytime socio-occupational dysfunction, which may be actual or perceived despite adequate sleep opportunities (4). According to the International classification of sleep disorders-3 edition (DSM-3) and the Chinese expert consensus on the assessment and management of stroke related sleep disorders (CEC-SSD) (5), stroke-related insomnia consists of two conditions: (i) post-stroke insomnia: insomnia first appears after stroke; (ii) stroke with insomnia: insomnia existing before stroke persists or worsens after stroke and meets the diagnostic criteria for insomnia. According to reports published 30 ~ 68% of poststroke patients were burdened with insomnia (6–8). In addition, insomnia is also thought to increase the risk of psychological problems (depression and anxiety), physical function (disability), and cognitive function (dementia) in IS patients (9, 10).
Strokes are more likely to occur in people who suffer from insomnia. The results of a meta-analysis of 160,867 patients in 15 studies showed that falling/maintaining asleep difficulty, and non-restorative sleep were positively strongly associated with the risk of stroke (11). According to a prospective clinical cohort study of stroke patients, insomnia patients were more likely to be depressed, anxious, disabled, and have difficulty returning to work than stroke patients without insomnia after 1 year after stroke (12). Surveys such as that conducted by Huang et al. (13) have shown that a negatively association between insomnia and the improvement in activities of daily living was found in subacute stroke inpatients. Tang et al. (14) showed that insomnia may make stroke survivors more susceptible to suicide. Kim et al. (10) demonstrated that insomnia had a negative effect on quality of life in stroke patients at the initial phases of rehabilitation. Also, insomnia IS patients have a higher recurrence stroke rate in the first year and a higher mortality rate within 6 years compared with no-insomnia IS patients (15).
Although the prevalence of insomnia continues to rise in many countries around the world (16, 17), patients often treat insomnia as a lifestyle issue rather than a major health problem (18). Also, the importance of insomnia diagnosis has been neglected in the routine medical examination of some IS patients. Clinician assessment and judgment based on the DSM-5 and CEC-SSD remain the criteria for diagnosing insomnia in stroke patients. Polysomnography (PSG) is considered as a common objective measure for the measurement of sleep disorders. However, PSG is not easy to obtain for most clinicians’ daily insomnia diagnosis routine (19), and is mostly used for the diagnosis of sleep disordered breathing (SDB). At the same time, PSG is very expensive for epidemiology and research, takes a long time and is not in line with clinical practice (19). However, multiple insomnia questionnaires have been developed including the PSQI, ISI, and AIS. All three questionnaires were used multiple times and translated into multiple languages and have been widely used in China. These questionnaires are considered to be efficient screening tools for insomnia. They are simple to perform and cost effective and do not demand additional special apparatus or facilities. The use of brief questionnaires for subjective assessment saves time and effort and ensures a high level of patient subjective willingness and compliance. Also, due to the self-explanatory nature of these tools, the need for and reliance on specialists and clinicians can be significantly reduced (20, 21). The three insomnia assessment tools used in this study, PSQI, ISI, and AIS, were not developed specifically for IS patients, and each scale validation was based on primary sleep disorders patients (22). Because of the significant role that the rating scales plays in the diagnosis of insomnia in IS patients, psychometric properties, cut-off values and diagnosis effectiveness of these tools need to be evaluated in IS patients. The purpose of this study was to the establish reliability and determine the optimal threshold values of PSQI, ISI, and AIS in the diagnosis of insomnia of IS patients.
2. Methods
2.1. Participants
This study recruited 569 IS patients on the inpatient medical care rosters of three general hospitals in Shanghai from January 2021 to September 2021. The three hospitals recruited 186, 165 and 218 IS patients, respectively. All patients received neuroradiological exam and the IS diagnosis were consistent with the “Chinese guidelines for diagnosis and treatment of acute ischemic stroke 2018 (23).” The following conditions were excluded:
unable to respond appropriately to questionnaires (such as aphasia);
transient ischemic attack (TIA) diagnosed by neurologists;
diagnosed with dementia, or another neurodegenerative or neurological condition.
The diagnosis of stroke-related insomnia (SRI) was determined by neurologist after evaluation and needed to meet both the diagnosis of stroke and insomnia. According to DSM-5 and CEC-SSD (5), the diagnosis of insomnia in this study was as follows. Patients who met all the following five conditions were insomnia IS patients, and the others were no-insomnia IS patients:
Patients complain of disgruntlement over sleep quality or quantity. They have one or more of the following symptoms: difficulty falling or maintaining sleep; wake up early and are unable to fall back asleep.
Patients complain that sleep disorders make them feel clinically significant distress or make it hard for them to do important things like socialize, work, behavioral, or other significant functions. They have one or more of the following symptoms: fatigue or lack of energy; decreased attention/concentration/memory; emotional instability; daytime fatigue; behavioral problems (a tendency to be hyperactive, impulsive, or aggressive); lack of energy or motivation; concerns about the quality of sleep.
These abnormalities cannot be accounted by inadequate sleep opportunities or inferior sleep environment.
Sleep disturbances and daytime symptoms ≥3 times a week.
The above symptoms are not explained by other sleep disorders.
In order to prevent the impacts of unstable neurological circumstances and environmental changes on the results, our investigation was conducted when the patient was conscious and exhibited stable vital signs after routine neurology treatment. The questionnaires of basic patient information, neurological function, three sleep assessment were applied at a mean of 5.79 days (SD = 2.34) after admission to the hospital. All 569 patients included in the study completed basic information collection and insomnia assessment using three scales (PSQI, ISI, AIS). The investigators conducted face-to-face interviews with IS patients at the neurology ward using PSQI, ISI, and AIS questionnaires. All questionnaires were evaluated when patients fully understood the contents of the questionnaire items.
Ethical approval was obtained from the Ethics Approval Committee of Shanghai Tenth People’s Hospital (Approval No. SHSY-IEC-KY-4.0/17–47/01). All participants agreed to participate in this study.
2.2. Measures and questionnaires
2.2.1. Basic information collection and functional assessment
The basic information of patients included age, sex, body mass index (BMI), marital status, education years, occupation, smoking, alcohol consumption, hypertension, diabetes, coronary heart disease. In this study, “smoke” was operationally defined as “current smokers” who had smoked within 30 days before the survey. “Drink” was operationally defined as consuming more than 15 g of alcohol per day within 30 days before the survey.
Each patient was assessed on the National Institute of Health stroke scale (NIHSS). The NIHSS was used to assess the severity of cerebral infarction and the degree of neurological deficit in patients. High NIHSS scores indicate more severe cerebral infarction, resulting in greater disability and functional decline. Also, each patient was assessed for activities of daily living (ADL) using Barthel index (BI). ADL competence is one of the most important indicators of the effectiveness of rehabilitation (24). The Barthel index (BI) is the most commonly used scale in the world to assess ADL competence (25). The BI consists of 10 items: feeding, bed and wheelchair transfer, personal hygiene, toileting, bathing, walking, walking up and down stairs, dressing, bowel control and urinary control. It is mainly suitable for detecting the changes of independent living activities of the elderly and patients before and after treatment.
2.2.2. PSQI
The PSQI has been extensively used in China to measure insomnia symptoms as an essential measure that is recommended globally (26). Based on DSM-5 and CEC-SSD, in clinical patients, the PSQI is the most widely used subjective measure of sleep dysfunction (27). The PSQI is a self-rating questionnaire, composed of 19 self-rated items (0 ~ 3 scale) with a total score from 0 to 21 assessing 7 domains of sleep quality and sleep disorders during the course of the past month: (i) subjective sleep quality (C1), (ii) sleep latency (C2), (iii) sleep duration (C3), (iv) normal sleep efficiency (C4), (v) sleep disturbances (C5), (vi) the application of sleep medication (C6), and (vii) daytime dysfunction (C7). The higher score indicates the worse sleep quality (28). A considerable amount of literature has been published on optimal threshold of PSQI in determining good or bad sleep quality. PSQI has not only been widely used in non-clinical populations, but has also been studied in a variety of clinical populations, including post-surgical patients (29), hemodialysis patients (30), schizophrenia (31), and anxiety (32). The cut-off values for the diagnosis of insomnia differ for different diseases. A cutoff score of 5 was suggested by Tsai et al. (33) to distinguish primary insomnia with Chinese version of PSQI among community-dwelling adults. A cross-sectional study of 327 IS patients suggested 8 as the cut-off score (34). The PSQI scale used in this study is the Chinese version translated by Liu et al. (35) in 1996. When the total PSQI score over 7, the sensitivity of distinguishing between normal and sleep-disordered populations was 98.3% and the specificity was 90.2% (35). The time to complete the PSQI assessment is approximately 5 to 10 min.
2.2.3. ISI
The ISI is a reliable instrument for population-level detection of insomnia cases and is sensitive to treatment response in clinical patients (36). The ISI is a seven-item, subjective assessment and screening tool that primarily assesses the character, gravity, and influence of insomnia over the previous 4 weeks (37). The items evaluate (i) difficulty falling asleep (initial) (I1a), (ii) difficulty maintaining sleep (middle) (I1b), (iii) early awakening (terminal) (I1c), (iv) satisfaction with current sleep situation (I2), (v) the degree of hindrance with routine function caused by sleep dilemmas (I3), (vi) the degree of impact or impairment on quality of life caused by the sleep difficulty (I4), and (vii) level of anxiety attributed to the sleep predicament (I5) (38). The time to complete the ISI assessment is approximately 5 min. The ISI uses a Likert scale of five points (0 ~ 4 scale), with a total score between 0 and 28. According to Reinsel (39), the best clinical cutoff score for identifying sleep disorders in breast cancer patients is 8. A cutoff score of 8 was suggested by Savard et al. (40) for identifying sleep difficulties in cancer patients, which produced 94.7% sensitivity and 47.4% specificity in cancer patients according to the DSM-IV criteria. At present, ISI has not established a cutoff score for the diagnosis of insomnia in many clinical patients (22).
2.2.4. AIS
The AIS was developed as a standard sleep assessment tool to quantify the degree of sleep difficulty, in accordance with the International Classification of Diseases-10 edition (ICD-10) (41). There are eight items: the first five are concerned with (i) time needed to fall asleep (A1), (ii) wakening during the night (A2), (iii) waking up earlier than desired time (A3), (iv) sleep duration (A4), and (v) sleep quality (A5); while the last three items concerned with (vi) daytime emotional state (A6), (vii) daytime functioning capacity (A7), and (viii) daytime sleepiness (A8). The AIS uses a Likert scale of 4 points (0 = no problem; 3 = serious problem), with an overall score from 0 to 24. Higher AIS scores represent higher levels of insomnia. The AIS has been translated into numerous languages and has been used by various populations worldwide, including English (41), Chinese (42), and Spanish (43). With a cut-off score of 6, AIS can be used for insomnia screening with the sensitivity of 93% and the specificity of 85% in Taiwanese cancer patients (42). A score of 6 was found to be the optimal cutoff when logistic regression analysis was performed between the total AIS score and ICD-10 insomnia diagnosis (44). Jeong et al. (45) also suggested the commonly accepted cut-off to be 6. In clinical practice and research, AIS can be utilized as a screening instrument to measure the severity of sleep-related issues as well as a tool in reliable insomnia screening. Although AIS is regarded as a useful insomnia screening tool for research prediction, it has not been tested on the IS patients.
2.3. Data analysis
EpiData 3.0 was used for data entry and aggregation for IS patients, and two researchers completed data input separately to ensure data accuracy. The IBM SPSS 22.0 and MedCalc 21.0 were used to implement statistical analysis. Kolmogorov–Smirnov test was used to test the normality of the data distribution, with p > 0.05 considered as normal distribution. Descriptive statistics were reported as mean ± standard deviation (SD) for variables with normal distributions and as median (interquartile range, IQR) variables with for skewed distributions. If the variables in each sample were normally distributed and the overall variance was equal, ANOVA was used to compare the means between the insomnia group and the non-insomnia group, and nonparametric test of Mann–Whitney U test was used if not. Chi-square test was used to compare the rates of categorical variables between the two groups.
ROC curve analysis was used to analyze the diagnostic validity of PSQI, ISI, and AIS. The area under ROC curve (AUC) was performed to evaluate the accuracy of PSQI, ISI, and AIS for the diagnosis of insomnia in IS patients. AUC is currently accepted as a diagnostic evaluation index and a method to determine the optimal diagnostic cutoff values. The value of AUC ranges from 0.5 to 1. The diagnostic value is acceptable when the score is between 0.71 and 0.8, excellent when it is 0.81–0.9, and outstanding when it is >0.9 (46). DeLong’s test (47) was performed to compare multiple ROC curves. Youden index was performed to determine the optimum cutoff diagnostic values of PSQI, ISI, and AIS (48). Cronbach’s α and McDonalds Ω coefficient was used for the reliability analysis of this study. The sensitivity, specificity, positive likelihood ratio (PLR), negative likelihood ratio (NLR), and diagnostic validity of the three scales were also analyzed. Cohen’s Kappa consistency was performed between insomnia diagnosis based on PSQI, ISI, and AIS and clinician-based insomnia diagnosis.
3. Results
3.1. Characteristics of IS patients
The Demographics of IS patients in the insomnia group and no-insomnia group in this study are shown in Table 1. A total of 569 patients were separated into insomnia group (n = 89) and no-insomnia group (n = 480) in accordance with the diagnostic criteria of insomnia in IS patients. In this study, 337 (59.23%) patients were male and 232 (40.77%) were female. The average age was 63.25 (SD = 10.34) and the average BMI was 24.27 (SD = 2.92). Additionally, 157 patients (27.59%) were employed before stroke. The median (IQR) of NIHSS and ADL scores of IS patients were 3 (3) and 55 (25). 415 (72.93%) IS patients have hypertension and 251 (44.11%) IS patients have diabetes. A comparative analysis of the insomnia group and the no-insomnia group revealed significant differences in terms of age, sex, BMI, and drinking. When the insomnia group was compared with the non-insomnia group, it was found that the insomnia group was older and included more female participants. Meanwhile, patients in the insomnia group had higher BMI and more alcohol consumption.
Table 1. Demographics of IS patients in the insomnia group and no-insomnia group (n = 569).
3.2. Analysis of PSQI, ISI, and AIS scores
The number of IS patients’ responses of the three questionnaires in the insomnia and no-insomnia groups is shown in Table 2. The median (IQR) of PSQI score in no-insomnia group was 3(2), and median (IQR) of PSQI score in insomnia group was 11(1). The median (IQR) of ISI score in no-insomnia group was 13(6), and median (IQR) of ISI score in insomnia group was 17(3). The median (IQR) of AIS score in no-insomnia group was 5(3), and median (IQR) of AIS score in insomnia group was 9(5). The results of the Mann–Whitney U test showed there were significant differences in scores of PSQI, ISI, and AIS between the insomnia group and the no-insomnia group (p < 0.001).
Table 2. The scores of PSQI, ISI, and AIS for insomnia group and no-insomnia group (N = 569).
3.3. Reliability of the PSQI, ISI, and AIS
The PSQI and ISI showed high reliability with Cronbach’s α-values of 0.875 and 0.858, while the AIS had an α coefficient of 0.734, demonstrating acceptable reliability. The factor analysis demonstrated that the ISI and AIS scale both emerged as a sole component (37, 41). The PSQI and ISI showed high reliability with McDonalds Ω values of 0.895 and 0.864, while the AIS had an α coefficient of 0.736, which is consistent with the Cronbach’s alpha coefficient.
3.4. Diagnostic validity of PSQI, ISI, and AIS
ROC curves of PSQI, ISI, and AIS in IS patients were showed in Figure 1. The AUCs of PSQI and ISI were 0.960 (95% CI: 0.946, 0.974) and 0.911 (95% CI: 0.882, 0.941), which were greater than 0.9, showing outstanding diagnostic ability of insomnia in IS patients. The AUC of AIS was 0.876 (95% CI: 0.837, 0.916), which ranged from 0.8 to 0.9, showing excellent diagnostic ability with PSQI and ISI in IS patients. DeLong’s test indicated that PSQI can be considered to have better diagnostic validity than ISI and IAS, while ISI and AIS cannot be differentiated regarding diagnostic validity (Table 3). The current study compared different PSQI, ISI, and AIS cutoff scores and the resulting sensitivity and specificity are shown in Figure 2. After determining the best cutoff values based on Youden’s index, which were 9 for PSQI, 15 for ISI and 8 for AIS, other statistics were further identified, such as sensitivity, specificity, Youden index, positive and negative likelihood ratio (Table 4). When it came to distinguishing between patients with and without insomnia, the PSQI had the highest sensitivity.
Figure 1. ROC curves of PSQI, ISI, and AIS in IS patients.
Table 3. DeLong’s test on pairwise comparison of ROC curves.
Figure 2. Sensitivity and specificity of PSQI, ISI, and AIS for various cutoff scores.
Table 4. Accuracy and diagnostic validity of PSQI, ISI, and AIS in IS patients.
Cohen’s Kappa test was used to compare the consistency between clinician-based insomnia diagnosis and questionnaire-based insomnia diagnosis in Table 4. In the questionnaire-based insomnia diagnosis, PSQI ≥9, ISI ≥ 15 and AIS ≥ 8 was considered as insomnia. Comparing PSQI with the clinician-based insomnia diagnosis revealed that Cohen’s kappa was 0.643 (95% CI: 0.570, 0.716). However, comparing ISI and AIS with insomnia diagnosis revealed that Cohen’s kappa was 0.459 (95% CI: 0.385, 0.533) and 0.539 (95% CI: 0.451, 0.627). PSQI can be considered to have more diagnostic concordance with clinicians than ISI (no CI overlap) while AIS has the same concordance as both PSQI and ISI (CI overlap).
4. Discussion
Insomnia is a common psychiatric complaint in IS patients (9). The reported prevalence of insomnia varies widely among studies due to the different definitions and assessment tools used. Among the 569 IS patients contained in this study, 89 were diagnosed as insomnia by clinicians. The prevalence of pre-stroke insomnia in IS patients was found to be 15.64% in this study, lower than that of previously reported levels. Surveys such as that conducted by Leppävuori et al. (49) have shown that 38.6% had insomnia before the stroke and 18.1% had insomnia after the stroke. Another study found a 12% rate of new-onset insomnia after stroke when excluding patients who had insomnia before stroke (50). A study from China found that the prevalence of insomnia in stroke patients was 57.9%, of which 32.2% had insomnia before the stroke and 25.7% had new insomnia after the stroke (51). A 4-year follow-up study of 21,438 insomniacs and 64,314 non-insomniacs found that the incidence of stroke was significantly higher in insomniacs than in non-insomniacs (incidence rate ratio = 1.85; 95% CI: 0.7, 2.05) and those insomniacs had 54% higher risk of developing stroke compared with non-insomniacs (adjusted hazard ratio = 1.54; 95% CI: 1.38, 1.72) (3). There is mounting clinical evidence to support a bidirectional relationship between insomnia and stroke. Insomnia is likely to be an independent risk factor for stroke, and stroke may also be a causative factor for the development of insomnia. Insomnia should be given more attention in IS patients.
This study demonstrated high sensitivity and specificity of the PSQI, ISI, and AIS in screening for insomnia in IS patients. It was discovered that the PSQI scale was the most sensitive (identifying insomnia in 100% of the IS patients when the cutoff value is 9), while DeLong’s test indicated that ISI and ASI were less but similarly sensitive, with ISI identifying insomnia in 91.01% of the IS patients when the cutoff value is 15, and AIS identifying insomnia in 75.28% of the IS patients when the cutoff value is 8. Some studies suggest that the diagnostic sensitivity to distinguish good sleep from poor sleep was 89.6% and the specificity was 86.5% when the total PSQI score > 5 (52). The Youden index of the PSQI was 0.852, with a strong ability to screen insomnia patients from non-insomnia patients. When the effect of morbidity was excluded, the positive likelihood ratio (PLR) for the PSQI was 6.761, which is a higher than ISI and AIS. Also, according to Charles, the ISI cutoff of 10 was the optimal score for identifying insomnia cases in the community people, with an 86.1% sensitivity and 87.7% specificity (36). When diagnosing insomnia on a score of ≥6, the AIS scale had a sensitivity of 93%, and a specificity of 85%, and the overall correct case identification rate is 90% (44). This confirmed the findings of these scales have excellent performance of these scales in screening for insomnia in IS patients in the literature (34, 36, 42). Our analysis showed that, identical to the widely used cutoff values for insomnia diagnosis in the general population in China, the best cutoff values in screening for insomnia in IS patients of the ISI questionnaires were “15.” However, the best cutoff values in screening for insomnia in IS patients of the PSQI and AIS questionnaires were “9” and “8” respectively, higher than the widely used diagnostic cutoff values for insomnia population in China.
Cohen’s kappa consistency statistic check was used to analyze the consistency of PSQI, ISI, and AIS with clinician-based insomnia diagnostic in IS patients. The kappa coefficient of PSQI, ISI, and AIS were 0.643, 0.459, and 0.539 and the asymptotic 95% confidence interval were (0.570, 0.716), (0.385, 0.533), and (0.451, 0.627). Landis and Koch (53) proposed that a kappa in the range of 0.41 ~ 0.60 be considered “moderate” agreement, kappa = 0.61 ~ 0.80 be considered “substantial” agreement, and kappa >0.81 be considered “almost perfect” agreement. This indicates that the consistency between PSQI with clinician-based insomnia diagnostic were substantial consistent. The consistency between ISI and AIS with clinician--based insomnia diagnostic was moderately consistent.
The PSQI focuses on the sleep status of patients in the past 4 weeks. Also, there are four types of responses (“not during the last month,” “less than once a week,” “once or twice a week,” “three or more times per week”) instead of using “very poor,” “poor,” “good,” “very good” and similar language to assess sleep. Such descriptions are important to accurately measure the severity of the IS patient’s sleep-related components. We believe this could help our patients better understand the PSQI question and choose the option that best matches their situation, compared to the AIS and ISI. In addition, these descriptions and response category of sleep quality severity can be translated into numerical rating ranges, so this may explain the greater sensitivity of PSQI in assessing insomnia in IS patients. PSQI is a widely used sleep quality evaluation tool to measure sleep quality and disturbances over the prior month and to discriminate between “good” and “poor” sleepers (22). Meanwhile, PSQI is often used together with the Epworth Sleepiness Scale (ESS) to assess daytime sleepiness, such as in medical students (54), COPD and asthma (55), or women with premenstrual syndrome (56).
Although the ROC curve was used to evaluate the diagnostic performance of the three questionnaires in insomnia, it was not the only thing to consider in choosing an assessment tool. The three questionnaires have similar content and language descriptions of sleep problems. Each of the three questionnaires has advantages and disadvantages when assessing insomnia. The PSQI cannot collect information about the patient’s need of treatment options and other aspects. The PSQI scale has more questions and takes longer to assess compared to the ISI and AIS, so it performs poorly in rapidly assessing the severity of insomnia in clinical patients. It should be emphasized that PSQI is the only one of the three questionnaires that needs to calculate and transform scores. Because of the need to integrate various responses and calculate such variables as sleep efficiency, hand-calculation of scores may be somewhat burdensome (22). This undoubtedly caused additional inconvenience and time consumption for clinical evaluation.
One advantage of ISI is that there are data based on which insomnia can be classified hierarchically. In a study which set out to investigate insomnia in long-term hospitalized older adults, Aluzaite et al. (57) classified ISI scores as follows: 0 ~ 7 points represent no insomnia; 8 ~ 14 points represent sub-threshold insomnia; 15 ~ 21 points represent moderate insomnia; 22 ~ 28 points represent severe insomnia. Under this score classification, the degree of insomnia in patients can be measured. Meanwhile, for PSQI and AIS such thresholds to measure the severity of insomnia in patients have not yet been investigated. In many clinical assessments, PSQI is often used as an index to measure the overall sleep quality, and ISI is used to measure the severity of insomnia (58). ISI focuses on the subjective symptoms of insomnia, the consequences and the degree of distress, and is sensitive to detect changes in sleep conditions brought about by treatment. However, researchers need to be alert to the possibility of false positive results when patients subjectively amplify their insomnia feelings.
The AIS is designed to assessment subjective feeling of sleep difficulty that only the subjective feelings of the interviewee are considered for the rating (41). In addition, the severity of the patient’s sleep difficulty as estimated by the interviewer, such as the hours of sleep or the approximate duration of sleep onset latency (41), was not considered in the AIS score since the interviewer’s judgment may not be consistent with the patient’s subjective assessment. Patients may exaggerate the negative feelings caused by short sleep duration, leading to bias in AIS assessment, which may also be the reason for the lower sensitivity of AIS.
There are several limitations to this study: All subjects were not tested with Polysomnography in this study. We compared the application of the three questionnaires in the diagnosis of insomnia, but did not give grade judgments for the severity of insomnia. Finally, because the information was gathered based on self-reporting, the data may be prone to information bias.
5. Conclusion
This study demonstrated high sensitivity and specificity and excellent diagnostic ability of the PSQI, ISI, and AIS questionnaires in screening for insomnia in IS patients. We found the best diagnostic cutoffs for PSQI, ISI, and AIS are ≥9, ≥15, and ≥8, respectively. Based on these cutoff values, the PSQI scale was revealed to be the most sensitive, while the ISI was the moderate sensitive and AIS was the least sensitive. Each of the three questionnaires has advantages and disadvantages when assessing insomnia. In the evaluation of insomnia in IS patients, the best questionnaire selection should be made according to the purpose of clinical evaluation (such as screening potential insomnia patients or the diagnosis of clinical insomnia) and considering the sensitivity and specificity of the questionnaire.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Institutional Review Boards of the Tenth People’s Hospital, Tongji University (Approval No. SHSY-IEC-KY-4.0/17–47/01). The patients/participants provided their written informed consent to participate in this study.
Author contributions
SN and YS: study conception and design, manuscript drafts and revisions. SN, SD, LCW, and LW: data collection. SN and QW: data analysis. SN, QW, SD, LCW, and YS: agree with manuscript results and conclusions. All authors approved the final manuscript and act as guarantors for the study.
Funding
This research was supported by the grants from the general project of National Natural Science Foundation of China (71774117).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Sierra, C, Coca, A, and Schiffrin, EL. Vascular mechanisms in the pathogenesis of stroke. Curr Hypertens Rep. (2011) 13:200–7. doi: 10.1007/s11906-011-0195-x
2. Kim, J, Thayabaranathan, T, Donnan, GA, Howard, G, Howard, VJ, Rothwell, PM, et al. Global stroke statistics 2019. Int J Stroke. (2020) 15:819–38. doi: 10.1177/1747493020909545
3. Wu, MP, Lin, HJ, Weng, SF, Ho, CH, Wang, JJ, and Hsu, YW. Insomnia subtypes and the subsequent risks of stroke: report from a nationally representative cohort. Stroke. (2014) 45:1349–54. doi: 10.1161/STROKEAHA.113.003675
4. Sateia, MJ. International classification of sleep disorders-third edition: highlights and modifications. Chest. (2014) 146:1387–94. doi: 10.1378/chest.14-0970
5. Beijing Neurology Association Sleep Disorder Committee. Chinese expert consensus on the assessment and management of stroke-related sleep disorders. Chin J Int Med. (2019) 58:17–26. doi: 10.3760/cma.j.issn.0578-1426.2019.01.004
6. Chen, YK, Lu, JY, Mok, VC, Ungvari, GS, Chu, WC, Wong, KS, et al. Clinical and radiologic correlates of insomnia symptoms in ischemic stroke patients. Int J Geriatr Psychiatry. (2011) 26:451–7. doi: 10.1002/gps.2547
7. Tang, WK, Grace, LC, Mok, V, Ungvari, GS, and Wong, KS. Insomnia and health-related quality of life in stroke. Top Stroke Rehabil. (2015) 22:201–7. doi: 10.1179/1074935714Z.0000000026
8. Sterr, A, Herron, K, Dijk, DJ, and Ellis, J. Time to wake-up: sleep problems and daytime sleepiness in long-term stroke survivors. Brain Inj. (2008) 22:575–9. doi: 10.1080/02699050802189727
9. Palomäki, H, Berg, A, Meririnne, E, Kaste, M, Lönnqvist, R, Lehtihalmes, M, et al. Complaints of poststroke insomnia and its treatment with mianserin. Cerebrovasc Dis. (2003) 15:56–62. doi: 10.1159/000067127
10. Kim, WH, Jung, HY, Choi, HY, Park, CH, Kim, ES, Lee, SJ, et al. The associations between insomnia and health-related quality of life in rehabilitation units at 1 month after stroke. J Psychosom Res. (2017) 96:10–4. doi: 10.1016/j.jpsychores.2017.02.008
11. He, Q, Zhang, P, Li, G, Dai, H, and Shi, J. The association between insomnia symptoms and risk of cardio-cerebral vascular events: a meta-analysis of prospective cohort studies. Eur J Prev Cardiol. (2017) 24:1071–82. doi: 10.1177/2047487317702043
12. Glozier, N, Moullaali, TJ, Sivertsen, B, Kim, D, Mead, G, Jan, S, et al. The course and impact of poststroke insomnia in stroke survivors aged 18 to 65 years: results from the psychosocial outcomes in StrokE (POISE) study. Cerebrovasc Dis Extra. (2017) 7:9–20. doi: 10.1159/000455751
13. Huang, RJ, Lai, CH, Lee, SD, Pai, FY, Chang, SW, Chung, AH, et al. Objective sleep measures in inpatients with subacute stroke associated with levels and improvements in activities of daily living. Arch Phys Med Rehabil. (2018) 99:699–706. doi: 10.1016/j.apmr.2017.12.016
14. Tang, WK, Lu, JY, Liang, H, Chan, TT, Mok, V, Ungvari, GS, et al. Is insomnia associated with suicidality in stroke? Arch Phys Med Rehabil. (2011) 92:2025–7. doi: 10.1016/j.apmr.2011.06.030
15. Li, LJ, Yang, Y, Guan, BY, Chen, Q, Wang, AX, Wang, YJ, et al. Insomnia is associated with increased mortality in patients with first-ever stroke: a 6-year follow-up in a Chinese cohort study. Stroke Vasc Neurol. (2018) 3:197–202. doi: 10.1136/svn-2017-000136
16. Kronholm, E, Partonen, T, Härmä, M, Hublin, C, Lallukka, T, Peltonen, M, et al. Prevalence of insomnia-related symptoms continues to increase in the Finnish working-age population. J Sleep Res. (2016) 25:454–7. doi: 10.1111/jsr.12398
17. Manzar, MD, Bekele, BB, Noohu, MM, Salahuddin, M, Albougami, A, Spence, DW, et al. Prevalence of poor sleep quality in the Ethiopian population: a systematic review and meta-analysis. Sleep Breath. (2020) 24:709–16. doi: 10.1007/s11325-019-01871-x
18. Sirajudeen, MS, Dilshad Manzar, M, Alqahtani, M, Alzhrani, M, Albougami, A, Somasekharan Pillai, P, et al. Psychometric properties of the Athens insomnia scale in occupational computer users. Healthcare. (2020) 8:8. doi: 10.3390/healthcare8020089
19. Fabbri, M, Beracci, A, Martoni, M, Meneo, D, Tonetti, L, and Natale, V. Measuring subjective sleep quality: a review. Int J Environ Res Public Health. (2021) 18:18. doi: 10.3390/ijerph18031082
20. Manzar, MD, Moiz, JA, Zannat, W, Spence, DW, Pandi-Perumal, SR, and Hussain, ME. Validity of the Pittsburgh sleep quality index in Indian university students. Oman Med J. (2015) 30:193–202. doi: 10.5001/omj.2015.41
21. Manzar, MD, BaHammam, AS, Hameed, UA, Spence, DW, Pandi-Perumal, SR, Moscovitch, A, et al. Dimensionality of the Pittsburgh sleep quality index: a systematic review. Health Qual Life Outcomes. (2018) 16:89. doi: 10.1186/s12955-018-0915-x
22. Omachi, TA. Measures of sleep in rheumatologic diseases: Epworth sleepiness scale (ESS), functional outcome of sleep questionnaire (FOSQ), insomnia severity index (ISI), and Pittsburgh sleep quality index (PSQI). Arthritis Care Res. (2011) 63:S287–96. doi: 10.1002/acr.20544
23. Chinese Neurology Society. Chinese guidelines for diagnosis and treatment of acute ischemic stroke 2018. Chin J Neurol. (2018) 51:662–83. doi: 10.3760/cma.j.issn.1006-7876.2018.09.004
24. Quinn, TJ, Dawson, J, Walters, MR, and Lees, KR. Functional outcome measures in contemporary stroke trials. Int J Stroke. (2009) 4:200–5. doi: 10.1111/j.1747-4949.2009.00271.x
25. Quinn, TJ, McArthur, K, Ellis, G, and Stott, DJ. Functional assessment in older people. BMJ. (2011) 343:d4681. doi: 10.1136/bmj.d4681
26. Buysse, D, Ancoli-Israel, S, Edinger, JD, Lichstein, KL, and Morin, CM. Recommendations for a standard research assessment of insomnia. Sleep. (2006) 29:1155–73. doi: 10.1093/sleep/29.9.1155
27. Mollayeva, T, Thurairajah, P, Burton, K, Mollayeva, S, Shapiro, CM, and Colantonio, A. The Pittsburgh sleep quality index as a screening tool for sleep dysfunction in clinical and non-clinical samples: a systematic review and meta-analysis. Sleep Med Rev. (2016) 25:52–73. doi: 10.1016/j.smrv.2015.01.009
28. Cole, JC, Motivala, SJ, Buysse, DJ, Oxman, MN, Levin, MJ, and Irwin, MR. Validation of a 3-factor scoring model for the Pittsburgh sleep quality index in older adults. Sleep. (2006) 29:112–6. doi: 10.1093/sleep/29.1.112
29. Toor, P, Kim, K, and Buffington, CK. Sleep quality and duration before and after bariatric surgery. Obes Surg. (2012) 22:890–5. doi: 10.1007/s11695-011-0541-8
30. Afsar, B, and Elsurer, R. The relationship between sleep quality and daytime sleepiness and various anthropometric parameters in stable patients undergoing hemodialysis. J Ren Nutr. (2013) 23:296–301. doi: 10.1053/j.jrn.2012.06.006
31. Ritsner, M, Kurs, R, Ponizovsky, A, and Hadjez, J. Perceived quality of life in schizophrenia: relationships to sleep quality. Qual Life Res. (2004) 13:783–91. doi: 10.1023/B:QURE.0000021687.18783.d6
32. Bush, AL, Armento, ME, Weiss, BJ, Rhoades, HM, Novy, DM, Wilson, NL, et al. The Pittsburgh sleep quality index in older primary care patients with generalized anxiety disorder: psychometrics and outcomes following cognitive behavioral therapy. Psychiatry Res. (2012) 199:24–30. doi: 10.1016/j.psychres.2012.03.045
33. Tsai, PS, Wang, SY, Wang, MY, Su, CT, Yang, TT, Huang, CJ, et al. Psychometric evaluation of the Chinese version of the Pittsburgh sleep quality index (CPSQI) in primary insomnia and control subjects. Qual Life Res. (2005) 14:1943–52. doi: 10.1007/s11136-005-4346-x
34. Xiao, M, Huang, G, Feng, L, Luan, X, Wang, Q, Ren, W, et al. Impact of sleep quality on post-stroke anxiety in stroke patients. Brain Behav. (2020) 10:e1716. doi: 10.1002/brb3.1716
35. Liu, X, Tang, M, and Hu, L. Reliability and validity of the Pittsburgh sleep quality index. Chin. J Psychiatry. (1996) 29:103–7.
36. Morin, CM, Belleville, G, Bélanger, L, and Ivers, H. The insomnia severity index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. (2011) 34:601–8. doi: 10.1093/sleep/34.5.601
37. Bastien, CH, Vallières, A, and Morin, CM. Validation of the insomnia severity index as an outcome measure for insomnia research. Sleep Med. (2001) 2:297–307. doi: 10.1016/S1389-9457(00)00065-4
38. Chen, D, Yin, Z, and Fang, B. Measurements and status of sleep quality in patients with cancers. Support Care Cancer. (2018) 26:405–14. doi: 10.1007/s00520-017-3927-x
39. Reinsel, RA, Starr, TD, O'Sullivan, B, Passik, SD, and Kavey, NB. Polysomnographic study of sleep in survivors of breast Cancer. J Clin Sleep Med. (2015) 11:1361–70. doi: 10.5664/jcsm.5264
40. Savard, MH, Savard, J, Simard, S, and Ivers, H. Empirical validation of the insomnia severity index in cancer patients. Psychooncology. (2005) 14:429–41. doi: 10.1002/pon.860
41. Soldatos, CR, Dikeos, DG, and Paparrigopoulos, TJ. Athens insomnia scale: validation of an instrument based on ICD-10 criteria. J Psychosom Res. (2000) 48:555–60. doi: 10.1016/S0022-3999(00)00095-7
42. Sun, JL, Chiou, JF, and Lin, CC. Validation of the Taiwanese version of the Athens insomnia scale and assessment of insomnia in Taiwanese cancer patients. J Pain Symptom Manag. (2011) 41:904–14. doi: 10.1016/j.jpainsymman.2010.07.021
43. Gómez-Benito, J, Ruiz, C, and Guilera, G. A Spanish version of the Athens insomnia scale. Qual Life Res. (2011) 20:931–7. doi: 10.1007/s11136-010-9827-x
44. Soldatos, CR, Dikeos, DG, and Paparrigopoulos, TJ. The diagnostic validity of the Athens insomnia scale. J Psychosom Res. (2003) 55:263–7. doi: 10.1016/S0022-3999(02)00604-9
45. Jeong, HS, Jeon, Y, Ma, J, Choi, Y, Ban, S, Lee, S, et al. Validation of the Athens insomnia scale for screening insomnia in south Korean firefighters and rescue workers. Qual Life Res. (2015) 24:2391–5. doi: 10.1007/s11136-015-0986-7
46. Abolkhair, AB, el-Kabbani, AO, al-Mulhem, A, AlFattani, AA, al-Hammadi, A, Alghamdi, H, et al. Psychometric and accuracy comparison of three commonly used questionnaires for the diagnosis of neuropathic pain. Saudi J Anaesth. (2021) 15:409–18. doi: 10.4103/sja.sja_352_21
47. DeLong, ER, DeLong, DM, and Clarke-Pearson, DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. (1988) 44:837–45. doi: 10.2307/2531595
48. Fluss, R, Faraggi, D, and Reiser, B. Estimation of the Youden index and its associated cutoff point. Biom J. (2005) 47:458–72. doi: 10.1002/bimj.200410135
49. Leppävuori, A, Pohjasvaara, T, Vataja, R, Kaste, M, and Erkinjuntti, T. Insomnia in ischemic stroke patients. Cerebrovasc Dis. (2002) 14:90–7. doi: 10.1159/000064737
50. Kim, KT, Moon, HJ, Yang, JG, Sohn, SI, Hong, JH, and Cho, YW. The prevalence and clinical significance of sleep disorders in acute ischemic stroke patients-a questionnaire study. Sleep Breath. (2017) 21:759–65. doi: 10.1007/s11325-016-1454-5
51. Zhang, X, and Guan, Q. Investigation of insomnia in stroke patients. Chin J Neurol. (2005) 38:324–5. doi: 10.3760/j.issn:1006‑7876.2005.05.013
52. Buysse, DJ, Reynolds, CR, Monk, TH, Berman, SR, and Kupfer, DJ. The Pittsburgh sleep quality index: a new instrument for psychiatric practice and research. Psychiatry Res. (1989) 28:193–213. doi: 10.1016/0165-1781(89)90047-4
53. Landis, JR, and Koch, GG. The measurement of observer agreement for categorical data. Biometrics. (1977) 33:159–74. doi: 10.2307/2529310
54. el Hangouche, AJ, Jniene, A, Aboudrar, S, Errguig, L, Rkain, H, Cherti, M, et al. Relationship between poor quality sleep, excessive daytime sleepiness and low academic performance in medical students. Adv Med Educ Pract. (2018) 9:631–8. doi: 10.2147/AMEP.S162350
55. Vukoja, M, Kopitovic, I, Milicic, D, Maksimovic, O, Pavlovic-Popovic, Z, and Ilic, M. Sleep quality and daytime sleepiness in patients with COPD and asthma. Clin Respir J. (2018) 12:398–403. doi: 10.1111/crj.12528
56. Conzatti, M, Perez, AV, Maciel, RF, De Castro, DH, Sbaraini, M, and Wender, M. Sleep quality and excessive daytime sleepiness in women with premenstrual syndrome. Gynecol Endocrinol. (2021) 37:945–9. doi: 10.1080/09513590.2021.1968820
57. Aluzaite, K, Wu, H, Hyslop, B, Young, J, Schultz, M, and Brockway, B. Perceived versus objective sleep quality in long-stay hospitalised older adults-a mixed methods study. Age Ageing. (2021) 50:955–62. doi: 10.1093/ageing/afaa292
58. Rubio-Arias, JÁ, Marín-Cascales, E, Ramos-Campo, DJ, Hernandez, AV, and Pérez-López, FR. Effect of exercise on sleep quality and insomnia in middle-aged women: a systematic review and meta-analysis of randomized controlled trials. Maturitas. (2017) 100:49–56. doi: 10.1016/j.maturitas.2017.04.003
Keywords: ischemic stroke, insomnia, diagnosis, Pittsburgh sleep quality index (PSQI), insomnia severity index (ISI), Athens insomnia scale (AIS)
Citation: Niu S, Wu Q, Ding S, Wu L, Wang L and Shi Y (2023) Comparison of three measures for insomnia in ischemic stroke patients: Pittsburgh sleep quality index, insomnia severity index, and Athens insomnia scale. Front. Neurol. 14:1118322. doi: 10.3389/fneur.2023.1118322
Edited by:
Haitham Jahrami, Arabian Gulf University, BahrainReviewed by:
Una Mikac, University of Zagreb, CroatiaTijana Bojić, University of Belgrade, Serbia
Lunliya Thampratankul, Mahidol University, Thailand
Copyright © 2023 Niu, Wu, Ding, Wu, Wang and Shi. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yan Shi, shlzk2002@163.com