Skip to main content

ORIGINAL RESEARCH article

Front. Cardiovasc. Med., 29 April 2022
Sec. Cardiac Rhythmology

Validation of Risk Scores for Predicting Atrial Fibrillation Detected After Stroke Based on an Electronic Medical Record Algorithm: A Registry-Claims-Electronic Medical Record Linked Data Study

  • 1Department of Neurology, Tainan Sin Lau Hospital, Tainan City, Taiwan
  • 2School of Pharmacy, Institute of Clinical Pharmacy and Pharmaceutical Sciences, College of Medicine, National Cheng Kung University, Tainan City, Taiwan
  • 3Division of Geriatrics, Department of Internal Medicine, Ditmanson Medical Foundation Chia-Yi Christian Hospital, Chiayi City, Taiwan
  • 4School of Medicine, National Taiwan University, Taipei City, Taiwan
  • 5Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
  • 6Heart & Brain Laboratory, Western University, London, ON, Canada
  • 7Department of Epidemiology and Biostatistics and Anatomy and Cell Biology, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
  • 8Robarts Research Institute, Western University, London, ON, Canada
  • 9Lawson Health Research Institute, London, ON, Canada
  • 10Division of Neurology, Department of Internal Medicine, Ditmanson Medical Foundation Chia-Yi Christian Hospital, Chiayi City, Taiwan
  • 11Department of Nursing, Min-Hwei Junior College of Health Care Management, Tainan City, Taiwan
  • 12Department of Pharmacy Systems, Outcomes and Policy, College of Pharmacy, University of Illinois at Chicago, Chicago, IL, United States

Background: Poststroke atrial fibrillation (AF) screening aids decisions regarding the optimal secondary prevention strategies in patients with acute ischemic stroke (AIS). We used an electronic medical record (EMR) algorithm to identify AF in a cohort of AIS patients, which were used to validate eight risk scores for predicting AF detected after stroke (AFDAS).

Methods: We used linked data between a hospital stroke registry and a deidentified database including EMRs and administrative claims data. EMR algorithms were constructed to identify AF using diagnostic and medication codes as well as free clinical text. Based on the optimal EMR algorithm, the incidence rate of AFDAS was estimated. The predictive performance of 8 risk scores including AS5F, C2HEST, CHADS2, CHA2DS2-VASc, CHASE-LESS, HATCH, HAVOC, and Re-CHARGE-AF scores, were compared using the C-index, net reclassification improvement, integrated discrimination improvement, calibration curve, and decision curve analysis.

Results: The algorithm that defines AF as any positive mention of AF-related keywords in electrocardiography or echocardiography reports, or presence of diagnostic codes of AF was used to identify AF. Among the 5,412 AIS patients without known AF at stroke admission, the incidence rate of AFDAS was 84.5 per 1,000 person-year. The CHASE-LESS and AS5F scores were well calibrated and showed comparable C-indices (0.741 versus 0.730, p = 0.223), which were significantly higher than the other risk scores.

Conclusion: The CHASE-LESS and AS5F scores demonstrated adequate discrimination and calibration for predicting AFDAS. Both simple risk scores may help select patients for intensive AF monitoring.

Introduction

Stroke is one of the leading causes of death and long-term disability in adults globally (1). In the past three decades, the absolute numbers of incident and prevalent strokes have increased by 70 and 85 percent, respectively (1). Moreover, the rate of stroke recurrence has remained unchanged over the last decade, particularly for cardioembolic strokes (2). Hence, current stroke guidelines highlight the importance of identifying potential sources of embolism to determine the optimal strategy for secondary stroke prevention (3).

Atrial fibrillation (AF) is the most common arrhythmia in clinical practice and a major source of cardioembolism. Oral anticoagulation can effectively reduce the risk of stroke recurrence in patients with known AF (KAF) (3). In stroke patients without KAF, applying sequential monitoring strategies including the use of implantable loop recorders could lead to an additional yield of 24% detection of AF after stroke (4), thereby allowing prompt secondary prevention. In the context of limited medical resources, it is reasonable to reserve aggressive poststroke AF search and electrocardiographic monitoring for selected patients at a high risk of AF detection based on their clinical characteristics (5).

To date, more than a dozen of risk scores have been developed or validated to assess the risk of AF detection after acute ischemic stroke (AIS) or transient ischemic attack (TIA) (6, 7). These risk scores vary considerably in their component variables, outcome definition, and ease of implementation (Supplementary Table 1). Besides patients’ demographics and comorbidities, many of these risk scores require additional diagnostic work-up to obtain the necessary components, such as blood, electrocardiography (ECG), and echocardiography markers (6). Due to the limitation of medical resources, not all risk scores could be applied equally in all healthcare settings by all practitioners. Moreover, most of currently available risk scores have not been externally validated or undergone head-to-head comparison (6), thus limiting their generalizability and clinical utility.

Therefore, this study aimed to assess the performance of 8 risk scores for predicting atrial fibrillation detected after stroke (AFDAS) based on routinely collected clinical variables.

Materials and Methods

Data Sources

This retrospective study was conducted in a 1,000-bed teaching hospital with approximately 650 stroke admissions annually. In this study, we used both data from the hospital stroke registry and the Ditmanson Research Database (DRD). Patients in both datasets are linked by unique identifiers. The hospital stroke center set up its stroke registry in 2007, conforming to the design of the nationwide Taiwan Stroke Registry (8). The stroke registry prospectively registers all cases of stroke admitted within 10 days of symptom onset. Data regarding the demographics, stroke etiology, risk factor profiles, stroke severity based on the National Institutes of Health Stroke Scale (NIHSS), acute treatment, and outcomes are routinely collected by trained stroke case managers. The DRD is a deidentified research-based database integrating electronic medical records (EMRs) and administrative claims data. It currently holds clinical information of over 1.4 million patients cared at the Ditmanson Medical Foundation Chia-Yi Christian Hospital from 2006 to 2021, including approximately 0.6 million inpatient records and 21.5 million outpatient records. We linked the stroke registry to the DRD using a unique encrypted patient identifier. Information regarding past medical history before admission and clinical variables collected during the index stroke hospitalization were obtained from the stroke registry. Billing information and medical records from 2 years before to 1 year after the index hospitalization were extracted from the DRD (Supplementary Figure 1).

The study protocol was approved by the Ditmanson Medical Foundation Chia-Yi Christian Hospital Institutional Review Board (IRB2020135). The requirement for informed consent was waived because of the retrospective design. The study protocol conforms to the ethical guidelines of the 1975 Declaration of Helsinki.

Study Population

The study population selection is shown in Supplementary Figure 2. The first hospitalization for AIS for each patient between Oct 2007 and Sep 2020 was identified from the stroke registry. Patients with an in-hospital stroke or whose data in the DRD could not be linked were excluded. The primary study outcome was AFDAS. In accordance with the updated definition of AFDAS, patients who had KAF before the index stroke hospitalization or whose admission ECG showed AF were further excluded (9). All patients were followed until the occurrence of AFDAS or death, the last visit within 1 year after the index stroke, or February 28, 2021, whichever occurred first.

Primary Outcome Ascertainment

Atrial fibrillation was identified in the EMR by adapting a previously validated AF-specific EMR algorithm based mainly on diagnostic and billing codes (10). The major modification was the use of natural language processing to detect the presence of keywords in physicians’ notes as well as ECG and echocardiography reports. Free text in the notes and reports were screened for mentions of AF-related keywords including “atrial fibrillation,” “AF,” “Afib,” and “PAF.” NegEx was used to determine whether the mention was positive or negative (11). The modified EMR algorithms comprise different components, which are further combined into composite algorithms as shown in Supplementary Table 2. By applying these algorithms, the AF status of each patient before discharge from the stroke hospitalization was determined using all available information up until the discharge date. We tested the discriminatory ability of each component and composite algorithm by using the manual screening of EMRs of 1,000 randomly selected patients (Supplementary Figure 2) as the gold standard. EMRs were reviewed by two experienced stroke neurologists. The degree of agreement of AF status between clinicians was assessed using the Kappa statistic. The performance of each algorithm was evaluated by estimating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We finally chose the best performing algorithm based on their area under the receiver operating characteristic curve (AUC) and we used it for identifying AFDAS in the whole cohort.

Risk Scores

We conducted a comprehensive search of the literature to identify articles reporting the derivation or validation of risk scores for predicting AFDAS published until December 2021. A total of 22 risk scores were identified (Supplementary Table 1). These risk scores vary in the types of component variables, which can be categorized into demographics, comorbidities, clinical markers, laboratory markers, ECG markers, echocardiography markers, and imaging markers. Considering the trade-offs between predictive performance and ease of implementation, we assessed 8 of the risk scores. They are based on readily available variables and are deemed suitable for routine clinical use (Supplementary Table 3).

The AS5F score was developed and validated using data from three prospective studies performing prolonged Holter-ECG monitoring in patients hospitalized for AIS or TIA (12). It is a very simple clinical score consisting of only age and NIHSS score. The C2HEST score was originally proposed to predict incident AF among Asian general populations (13). It demonstrated moderately good predictive abilities in predicting the risk of incident AF as well as death and hospitalization in patients with heart failure with preserved ejection fraction (14). It also performed well in discriminating the risk of developing incident AF in a French population hospitalized for AIS (15). The well-known CHADS2 and CHA2DS2-VASc scores were formerly developed for predicting stroke in patients with AF (16, 17) and were later found to predict incident AF in a large cohort of patients hospitalized for AIS collected from the French national administrative database (18). The CHASE-LESS score was developed by using data from patients hospitalized for AIS from the Taiwan National Health Insurance claims database and included both positive and negative items in the score calculation (19). The HATCH score was initially developed to identify patients who are likely to progress from paroxysmal to persistent forms of AF within 1 year (20) and was later found to predict new-onset AF in the general population (21) and patients with AIS (22). The HAVOC score was established by using data from a hospital-based research database where predictor and outcome variables were identified from diagnostic codes (23). It only included patients with cryptogenic stroke or TIA in the study cohort. The final risk model is a modification of the original CHARGE-AF model, which was intended to predict incident AF in the general population (24). The Re-CHARGE-AF model exhibited an excellent discrimination and good calibration in a cohort of AIS patients (7).

Statistical Analysis

Categorical variables were described with counts and percentages. Continuous variables were reported as means with standard deviations for normally distributed data and the median and interquartile range for non-normally distributed data. Differences between groups were compared by Chi-square tests for categorical variables and t-tests or Mann–Whitney U tests for continuous variables, as appropriate.

The incidence rate of AFDAS was expressed as events per 1,000 person-years. To assess the prediction performance of each risk score, Cox proportional hazard regression analyses were performed by entering each risk score as a continuous variable. After fitting the model, the proportional hazards assumption was assessed using the Schoenfeld test. Harrell’s concordance index (C-index) was calculated for evaluation and comparison of model performance. The optimism-adjusted C-index was also estimated using bootstrapping with 250 replications of patients sampled with replacement. The value of C-index ranges from 0.5 to 1.0, with 0.5 indicating random chance and 1 indicating perfect concordance. In clinical application, a C-index value of 0.7 or higher is considered acceptable discrimination (25). In addition to the C-index, the improvement in predictive performance was assessed by calculating the continuous net reclassification improvement (NRI) and integrated discrimination improvement (IDI) indices (26, 27). Unlike categorical NRI, continuous NRI does not require prespecified risk categories. It measures upward and downward changes in an event’s predicted probabilities. The IDI index represents the difference in discrimination slopes, which quantity the difference between the average of the predicted probabilities of an event for those with events and the corresponding average for those without events (26, 27). Higher NRI and IDI values indicate better discrimination.

Calibration was assessed by plotting the predicted versus actual risk of AFDAS for quintiles of the predicted probability. Bootstraps with 1,000 times of resamples with replacement were applied to the calibration curve. In a well calibrated model, the data points in the calibration curve should be close to the 45-degree diagonal line. Finally, to assess the clinical usefulness of the risk scores, decision curve analysis was used to estimate the net benefit, i.e., the ability to make better decisions with a model than without it (28).

All statistical analyses were performed using Stata 15.1 (StataCorp, College Station, TX, United States) and R version 4.1.1 (R Foundation for Statistical Computing, Vienna, Austria). Pairwise comparisons of C-indices were performed with the STATA procedures “somersd” and “lincom” (29). The optimism-adjusted C-index was estimated using the R package “rms”. The NRI and IDI indices were calculated using the R package “survIDINRI”. The calibration plot was performed using the R package “riskRegression.” Two-tailed p-values were considered statistically significant at <0.05.

Results

Optimal Atrial Fibrillation-Specific Electronic Medical Record Algorithm

Of the 1,000 patients whose EMRs were manually reviewed, 169 patients were adjudicated as having AF. The Kappa statistic between the reviewers was 0.91 indicating excellent interrater agreement. Supplementary Table 4 lists the performance of various AF-specific EMR algorithms. The highest AUC (0.990) was attained by the composite algorithm that defines AF as any positive mention of AF-related keywords in ECG or echocardiography reports, or presence of diagnostic codes of AF (AF-D in Supplementary Tables 2, 4). Its sensitivity, specificity, PPV, and NPV were 98.2, 99.8, 98.8, and 99.6%, respectively. Therefore, this algorithm was chosen for identifying KAF and AFDAS in the whole cohort.

Characteristics of the Study Population

We identified 6,523 patients hospitalized for AIS from the stroke registry (Supplementary Figure 2). After excluding 199 patients with in-hospital stroke and 2 patients with unavailable DRD data, the remaining 6,322 patients were successfully linked to the DRD. Among them, 910 patients were determined to have KAF at stroke admission and were thus excluded from the study population. During the follow-up of the remaining 5,412 patients, 316 (5.8%) patients were identified as having AFDAS after a median follow-up of 10.6 months. Each patient had a mean of 3.1 hospital visits per month during the follow-up period. The incidence rate of AFDAS was 84.5 per 1,000 person-year. Table 1 shows the characteristics of the study population. Patients with AFDAS were older, more likely to be female, and tended to have coronary artery disease, prior stroke or TIA, peripheral artery disease, valve disease, and chronic obstructive pulmonary disease, as well as lower diastolic blood pressure, height, weight, and body mass index than those with no AFDAS. They also had significantly higher NIHSS score and AFDAS risk scores but were less likely to have diabetes and hyperlipidemia.

TABLE 1
www.frontiersin.org

Table 1. Baseline characteristics of the study population.

Performance of Risk Scores

Table 2 shows the performance of the eight risk scores. All the risk scores were significantly associated with the risk of AFDAS. However, their C-indices varied considerably, ranging from 0.584 to 0.741. Only the CHASE-LESS and AS5F scores achieved a C-index value greater than 0.7. Supplementary Table 5 shows the p-values of pairwise comparisons of C-indices between the risk scores. The C-indices of the CHASE-LESS and AS5F scores were comparable (0.741 versus 0.730, p = 0.223) but were significantly higher than those of the C2HEST, CHADS2, CHA2DS2-VASc, HATCH, HAVOC, and Re-CHARGE-AF scores (all p < 0.001 except for AS5F versus Re-CHARGE-AF p = 0.002). Both NRI and IDI analyses (Table 3) demonstrated that the CHASE-LESS and AS5F scores more accurately predicted AFDAS than did the other 6 risk scores. The CHASE-LESS score significantly improved reclassification over the C2HEST, CHADS2, CHA2DS2-VASc, HATCH, HAVOC, and Re-CHARGE-AF scores with NRI values of 27.1, 26.6, 24.5, 32.7, 30.5, and 20.2%, respectively. The corresponding NRI values for the AS5F score were 29.9, 37.4, 28.7, 34.9, 35.3, and 18.8%, respectively. The IDI indices also showed significant improvement in discrimination slopes for the CHASE-LESS and AS5F scores compared to the other 6 risk scores.

TABLE 2
www.frontiersin.org

Table 2. Performance of risk scores for prediction of atrial fibrillation detected after stroke.

TABLE 3
www.frontiersin.org

Table 3. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) indices between each pair of two different risk scores.

The calibration plots (Figure 1) for the AS5F, CHASE-LESS, and Re-CHARGE-AF scores displayed a close agreement between the predicted and estimated actual risks, indicating good calibration. By contrast, the calibration curve of the C2HEST, CHADS2, CHA2DS2-VASc, HATCH, and HAVOC scores demonstrated a substantial deviation from the 45-degree diagonal line. The decision curve analysis graph (Figure 2) shows that the CHASE-LESS and AS5F scores provide a larger net benefit across ranges of threshold probabilities than the other risk scores. The CHASE-LESS score adds more benefit than the AS5F score if the threshold probability is between approximately 7.5 and 30% whereas the AS5F score has a higher net benefit if the threshold probability is between approximately 4 and 7.5%.

FIGURE 1
www.frontiersin.org

Figure 1. Calibration plots for the eight risk scores for predicting the risk of atrial fibrillation detected after stroke within 1 year of stroke.

FIGURE 2
www.frontiersin.org

Figure 2. Decision curve analysis graph for the risk scores. The y-axis represents the net benefit, and the x-axis represents the threshold probability. The diagonal blue line (screen all) indicates the net benefit assuming all patients are at risk for atrial fibrillation detected after stroke, while the horizontal brown line (screen none) assumes none are at risk for atrial fibrillation detected after stroke.

Discussion

We assessed the performance of eight risk scores for predicting the risk of AFDAS and found that the CHASE-LESS and AS5F scores performed equally well and significantly better than the C2HEST, CHADS2, CHA2DS2-VASc, HATCH, HAVOC, and Re-CHARGE-AF scores.

Performance of Risk Scores for Atrial Fibrillation Detected After Stroke

Supplementary Table 1 summarized the existing risk scores for predicting AFDAS. Although some risk scores achieved excellent performance with a C-index of 0.8 or higher in the derivation sample (3035), most of them were not tested outside the derivation sample. Moreover, these scores need additional information, such as blood, echocardiography, and imaging markers, that may require extra effort to procure. Consequently, these risk scores are not very practical for routine clinical use.

On the other hand, despite being relatively easy to implement, six of the assessed risk scores in this study including the C2HEST, CHADS2, CHA2DS2-VASc, HATCH, HAVOC, and Re-CHARGE-AF scores had only modest predictive performance. Their C-indices were below the accepted 0.7 threshold necessary to have clinical relevance at the individual patient level. Previous studies have shown that the C-indices of the CHADS2 score ranged from 0.536 to 0.700 (12, 18, 19, 22, 36, 37) and those of the CHA2DS2-VASc score varied between 0.578 and 0.706 (15, 18, 19, 22, 36, 37), whereas the HATCH score attained C-indices of 0.612 and 0.653 (22). Only the C2HEST score achieved a C-index of 0.734 in the external validation cohort (15). This might be because all the six scores do not consider stroke severity, which was found to predict the diagnosis of AFDAS and improve the performance of risk scores (6, 22). By contrast, the CHASE-LESS and AS5F scores include the NIHSS score in the risk score formula and indeed performed satisfactorily.

Furthermore, most of the risk scores were not derived from a stroke population and some of them were originally developed for a different purpose. Consequently, the relationships between the predictors and outcome may be dissimilar across contexts of risk score application. For example, diabetes mellitus, one of the components of the CHADS2 and CHA2DS2-VASc scores, is in fact a negative rather than positive predictor of AFDAS (7, 19). Such a negative association was also observed in the case of incident AF after embolic stroke of undetermined source (38). As a result, both scores were found not very useful in predicting AFDAS (22, 36, 39). By contrast, diabetes mellitus has a negative coefficient in the CHASE-LESS and Re-CHARGE-AF scores, which might contribute to their better calibration.

Clinical Significance of Atrial Fibrillation Detected After Stroke

Atrial fibrillation detected after stroke is a novel and distinct clinical entity. Compared to those with KAF, stroke patients with AFDAS have fewer vascular risk factors and comorbidities, fewer structural abnormalities of the heart such as left atrial enlargement, and are more likely to have strokes affecting the insular cortex (9). In addition to previously undiagnosed cardiogenic AF leading to stroke, stroke itself can cause transient neurogenic AF through inflammation, autonomic dysfunction, and stroke-induced heart injury (9, 40). Some AFDAS may even be just a bystander, rather than the cause of the stroke (41). However, we were unable to differentiate the phenotypes of AFDAS in the current study.

Although the risk of stroke recurrence of AFDAS is 26% lower than KAF (42), its risk is twice as high as the risk of stroke recurrence of no-AF (sinus rhythm or other non-AF rhythms) (9). The different risk of stroke recurrence between KAF and AFDAS may be because AFDAS is a heterogeneous mixture of high-burden (mainly cardiogenic) and low-burden (mainly neurogenic) AF phenotypes (40). In addition to stroke recurrence, AFDAS plays a significant role in many deleterious clinical outcomes. For example, AFDAS confirmed either during the stroke admission or during the outpatient follow-up period after discharge increases the risk of dementia by 78 and 74%, respectively, compared to no-AF (43). Furthermore, AFDAS raises the risk of all-cause death by 60% than no-AF (44).

Application of Risk Scores

Poststroke AF screening is one of the research priorities identified by global experts regarding the detection of undiagnosed AF (45). The detection of AF poststroke generally dictates the necessity of long-term anticoagulation (46). The diagnosis of AF in the current study, as well as in many of the prior studies (Supplementary Table 1), was made in usual-care settings rather than clinical trials. In other words, AF was mostly documented using 12-lead ECG or 24-h Holter ECG. As such, the findings of this study are valid for relatively high-burden AFDAS detected on short-term monitoring rather than long-term strategies (47). Furthermore, this phenotype of AFDAS is likely to be more clinically relevant and benefits more from oral anticoagulation than low-burden AFDAS (41). Even though intensive or prolonged monitoring may greatly increase the yield of identification of subclinical AF (48), whether prolonged cardiac monitoring can decrease the risk of stroke recurrence is yet to be determined (49).

In view of this and to be resource efficient, intensive or prolonged ECG monitoring should be reserved for a selected population at a high risk of AFDAS. The European Society of Cardiology guidelines (50) also recommend that not all stroke patients are expected to benefit from prolonged ECG monitoring. Instead, they should be reserved for selected stroke patients without previous KAF such as those bearing a high risk of developing AF. Consequently, development and validation of AFDAS prediction tools are of high clinical significance and relevance. Some of the risk score development studies also established cutoff points to stratify patients into different risk groups (12, 19, 23), even though risk stratification is not meant as absolute criteria for selecting candidates for advanced cardiac monitoring (12). This study externally validated eight existing risk scores that are based on simple clinical parameters. The study findings can provide guidance to physicians in making clinical decisions in the context of medical resource allocation and patient preference, particularly in healthcare systems where the access to advanced cardiac monitoring is limited.

Study Strengths and Limitations

This study performed a head-to-head comparison of eight AFDAS predicting scores widely used in clinical practice by using a large cohort of stroke patients with robustly validated outcomes. Some limitations are worth noting. First, patients in this study were traced through the hospital database. Because patients might receive a diagnosis of AF outside the study hospital, some degree of outcome misclassification is possible. Nevertheless, the relatively frequent visits to the study hospital (>3 visits per month) by the patients may have mitigated this problem. Second, this is a single-site study and therefore the study findings may not be generalizable to other settings. Further validation in other datasets, preferably prospective cohorts of patients, is recommended. Third, the methods used to identify poststroke AF were not standardized in all patients and therefore biases in the measurement of the outcome might exist. Nevertheless, this study reflected real-world clinical practice where stroke patients are generally diagnosed with AF using 12-lead ECG or 24-h Holter ECG. Fourth, the CHASE-LESS score may have performed better than other scores in this cohort because it was developed in a population from Taiwan, with similar sociodemographic and ethnic characteristics.

Conclusion

Of the eight risk scores validated in this cohort, the CHASE-LESS and AS5F scores demonstrated adequate discrimination and calibration for predicting AFDAS. These two simple risk scores may help refine the patient population for whom intensive AF monitoring is most likely to be beneficial. Further comparisons in AIS cohorts with different characteristics are needed.

Data Availability Statement

The data analyzed in this study is subject to the following licenses/restrictions: The data used in this study cannot be made available because of restrictions regarding the use of EMR data. Requests to access these datasets should be directed to S-FS, richard.sfsung@gmail.com.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ditmanson Medical Foundation Chia-Yi Christian Hospital Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

C-YH, H-MK, and S-FS contributed to the conception and design of the study. K-LS and S-FS organized the database and performed the statistical analysis. C-YH and H-MK wrote the first draft of the manuscript. K-LS, LS, S-FS, and S-JL wrote the sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

This research was supported in part by the Ditmanson Medical Foundation Chia-Yi Christian Hospital-National Chung Cheng University Joint Research Program (grant number CYCH-CCU-2022-03). This study is based in part on data from the Ditmanson Research Database (DRD) provided by Ditmanson Medical Foundation Chia-Yi Christian Hospital. The interpretation and conclusions contained herein do not represent the position of Ditmanson Medical Foundation Chia-Yi Christian Hospital.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors thank the help from the Clinical Data Center, Ditmanson Medical Foundation Chia-Yi Christian Hospital for providing administrative and technical support.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.888240/full#supplementary-material

References

1. GBD 2019 Stroke Collaborators. Global, regional, and national burden of stroke and its risk factors, 1990–2019: a systematic analysis for the global burden of disease study 2019. Lancet Neurol. (2021) 20:795–820. doi: 10.1016/s1474-4422(21)00252-0

CrossRef Full Text | Google Scholar

2. Flach C, Muruet W, Wolfe CDA, Bhalla A, Douiri A. Risk and secondary prevention of stroke recurrence. Stroke. (2020) 51:2435–44. doi: 10.1161/strokeaha.120.028992

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kleindorfer DO, Towfighi A, Chaturvedi S, Cockroft KM, Gutierrez J, Lombardi-Hill D, et al. 2021 guideline for the prevention of stroke in patients with stroke and transient ischemic attack: a guideline from the American heart association/American stroke association. Stroke. (2021) 52:e364–467. doi: 10.1161/str.0000000000000375

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Sposato LA, Cipriano LE, Saposnik G, Vargas ER, Riccio PM, Hachinski V. Diagnosis of atrial fibrillation after stroke and transient ischaemic attack: a systematic review and meta-analysis. Lancet Neurol. (2015) 14:377–87. doi: 10.1016/s1474-4422(15)70027-x

CrossRef Full Text | Google Scholar

5. Schnabel RB, Haeusler KG, Healey JS, Freedman B, Boriani G, Brachmann J, et al. Searching for atrial fibrillation poststroke: a white paper of the AF-SCREEN international collaboration. Circulation. (2019) 140:1834–50. doi: 10.1161/circulationaha.119.040267

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Kishore AK, Hossain MJ, Cameron A, Dawson J, Vail A, Smith CJ. Use of risk scores for predicting new atrial fibrillation after ischemic stroke or transient ischemic attack—a systematic review. Int J Stroke. (2021). doi: 10.1177/17474930211045880 [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Ashburner JM, Wang X, Li X, Khurshid S, Ko D, Lipsanopoulos AT, et al. Re-CHARGE-AF: recalibration of the CHARGE-AF model for atrial fibrillation risk prediction in patients with acute stroke. J Am Heart Assoc. (2021) 10:e022363. doi: 10.1161/jaha.121.022363

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Hsieh F-I, Lien L-M, Chen S-T, Bai C-H, Sun M-C, Tseng H-P, et al. Get with the guidelines-stroke performance indicators: surveillance of stroke care in the Taiwan stroke registry. Circulation. (2010) 122:1116–23. doi: 10.1161/circulationaha.110.936526

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Sposato LA, Chaturvedi S, Hsieh C-Y, Morillo CA, Kamel H. Atrial fibrillation detected after stroke and transient ischemic attack: a novel clinical concept challenging current views. Stroke. (2022) 53:e94–103. doi: 10.1161/strokeaha.121.034777

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Khurshid S, Keaney J, Ellinor PT, Lubitz SA. A simple and portable algorithm for identifying atrial fibrillation in the electronic medical record. Am J Cardiol. (2016) 117:221–5. doi: 10.1016/j.amjcard.2015.10.031

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Chapman WW, Bridewell W, Hanbury P, Cooper GF, Buchanan BG. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform. (2001) 34:301–10. doi: 10.1006/jbin.2001.1029

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Uphaus T, Weber-Krüger M, Grond M, Toenges G, Jahn-Eimermacher A, Jauss M, et al. Development and validation of a score to detect paroxysmal atrial fibrillation after stroke. Neurology. (2019) 92:e115–24. doi: 10.1212/wnl.0000000000006727

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Li Y-G, Pastori D, Farcomeni A, Yang P-S, Jang E, Joung B, et al. A simple clinical risk score (C2HEST) for predicting incident atrial fibrillation in Asian subjects: derivation in 471,446 Chinese subjects, with internal validation and external application in 451,199 Korean subjects. Chest. (2019) 155:510–8. doi: 10.1016/j.chest.2018.09.011

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Liang W, Wu Y, Xue R, Wu Z, Wu D, He J, et al. C2HEST score predicts clinical outcomes in heart failure with preserved ejection fraction: a secondary analysis of the TOPCAT trial. BMC Med. (2021) 19:44. doi: 10.1186/s12916-021-01921-w

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Li Y-G, Bisson A, Bodin A, Herbert J, Grammatico-Guillon L, Joung B, et al. C2 HEST score and prediction of incident atrial fibrillation in poststroke patients: a French nationwide study. J Am Heart Assoc. (2019) 8:e012546. doi: 10.1161/jaha.119.012546

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of clinical classification schemes for predicting stroke: results from the national registry of atrial fibrillation. JAMA. (2001) 285:2864–70. doi: 10.1001/jama.285.22.2864

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Lip GYH, Nieuwlaat R, Pisters R, Lane DA, Crijns HJGM. Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the euro heart survey on atrial fibrillation. Chest. (2010) 137:263–72. doi: 10.1378/chest.09-1584

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Fauchier L, Clementy N, Pelade C, Collignon C, Nicolle E, Lip GYH. Patients with ischemic stroke and incident atrial fibrillation: a nationwide cohort study. Stroke. (2015) 46:2432–7. doi: 10.1161/strokeaha.115.010270

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Hsieh C-Y, Lee C-H, Sung S-F. Development of a novel score to predict newly diagnosed atrial fibrillation after ischemic stroke: the CHASE-LESS score. Atherosclerosis. (2020) 295:1–7. doi: 10.1016/j.atherosclerosis.2020.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

20. de Vos CB, Pisters R, Nieuwlaat R, Prins MH, Tieleman RG, Coelen R-JS, et al. Progression from paroxysmal to persistent atrial fibrillation clinical correlates and prognosis. J Am Coll Cardiol. (2010) 55:725–31. doi: 10.1016/j.jacc.2009.11.040

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Suenari K, Chao T-F, Liu C-J, Kihara Y, Chen T-J, Chen S-A. Usefulness of HATCH score in the prediction of new-onset atrial fibrillation for Asians. Medicine. (2017) 96:e5597. doi: 10.1097/md.0000000000005597

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Hsieh C-Y, Lee C-H, Wu DP, Sung S-F. Prediction of new-onset atrial fibrillation after first-ever ischemic stroke: a comparison of CHADS2, CHA2DS2-VASc and HATCH scores and the added value of stroke severity. Atherosclerosis. (2018) 272:73–9. doi: 10.1016/j.atherosclerosis.2018.03.024

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Kwong C, Ling AY, Crawford MH, Zhao SX, Shah NH. A clinical score for predicting atrial fibrillation in patients with cryptogenic stroke or transient ischemic attack. Cardiology. (2017) 138:133–40. doi: 10.1159/000476030

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Alonso A, Krijthe BP, Aspelund T, Stepas KA, Pencina MJ, Moser CB, et al. Simple risk model predicts incidence of atrial fibrillation in a racially and geographically diverse population: the CHARGE-AF consortium. J Am Heart Assoc. (2013) 2:e000102. doi: 10.1161/jaha.112.000102

PubMed Abstract | CrossRef Full Text | Google Scholar

25. LaValley MP. Logistic regression. Circulation. (2008) 117:2395–9. doi: 10.1161/circulationaha.106.682658

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Pencina MJ, D’Agostino RB, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. (2008) 27:157–72; discussion 207–12. doi: 10.1002/sim.2929

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Pencina MJ, D’Agostino RB, Steyerberg EW. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers. Stat Med. (2011) 30:11–21. doi: 10.1002/sim.4085

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989x06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Newson RB. Comparing the predictive powers of survival models using Harrell’s C or Somers’. D. Stata J. (2010) 10:339–58. doi: 10.1177/1536867X1001000303

CrossRef Full Text | Google Scholar

30. Suissa L, Bertora D, Lachaud S, Mahagne MH. Score for the targeting of atrial fibrillation (STAF): a new approach to the detection of atrial fibrillation in the secondary prevention of ischemic stroke. Stroke. (2009) 40:2866–8. doi: 10.1161/strokeaha.109.552679

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Yoshioka K, Watanabe K, Zeniya S, Ito Y, Hizume M, Kanazawa T, et al. A score for predicting paroxysmal atrial fibrillation in acute stroke patients: iPAB score. J Stroke Cerebrovasc Dis. (2015) 24:2263–9. doi: 10.1016/j.jstrokecerebrovasdis.2015.06.019

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Seo W-K, Kang S-H, Jung J-M, Choi J-Y, Oh K. Novel composite score to predict atrial fibrillation in acute stroke patients: AF predicting score in acute stroke. Int J Cardiol. (2016) 209:184–9. doi: 10.1016/j.ijcard.2016.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Muscari A, Bonfiglioli A, Faccioli L, Ghinelli M, Magalotti D, Manzetto F, et al. Usefulness of the MrWALLETS scoring system to predict first diagnosed atrial fibrillation in patients with ischemic stroke. Am J Cardiol. (2017) 119:1023–9. doi: 10.1016/j.amjcard.2016.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Muscari A, Barone P, Faccioli L, Ghinelli M, Trossello MP, Puddu GM, et al. Usefulness of the ACTEL score to predict atrial fibrillation in patients with cryptogenic stroke. Cardiology. (2020) 145:168–77. doi: 10.1159/000505262

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Ntaios G, Perlepe K, Lambrou D, Sirimarco G, Strambo D, Eskandari A, et al. Identification of patients with embolic stroke of undetermined source and low risk of new incident atrial fibrillation: the AF-ESUS score. Int J Stroke. (2020) 16:29–38. doi: 10.1177/1747493020925281

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Liu R, Yang X, Li S, Jiang Y, Wang Y, Wang Y. Novel composite scoring system to predict unknown atrial fibrillation in acute ischemic stroke patients. Brain Res. (2017) 1674:36–41. doi: 10.1016/j.brainres.2017.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Chen Y-L, Wang H-T, Chen H-C, Liu W-H, Hsueh S, Chung W-J, et al. A risk stratification scoring system for new-onset atrial fibrillation after ischemic stroke. Medicine. (2020) 99:e20881. doi: 10.1097/md.0000000000020881

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Bahit MC, Sacco RL, Easton JD, Meyerhoff J, Cronin L, Kleine E, et al. Predictors of development of atrial fibrillation in patients with embolic stroke of undetermined source: an analysis of the RE-SPECT ESUS trial. Circulation. (2021) 144:1738–46. doi: 10.1161/circulationaha.121.055176

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Baturova MA, Lindgren A, Carlson J, Shubik YV, Olsson SB, Platonov PG. Predictors of new onset atrial fibrillation during 10-year follow-up after first-ever ischemic stroke. Int J Cardiol. (2015) 199:248–52. doi: 10.1016/j.ijcard.2015.07.047

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Cerasuolo JO, Cipriano LE, Sposato LA. The complexity of atrial fibrillation newly diagnosed after ischemic stroke and transient ischemic attack: advances and uncertainties. Curr Opin Neurol. (2017) 30:28–37. doi: 10.1097/wco.0000000000000410

CrossRef Full Text | Google Scholar

41. Sposato LA, Andrade J. Prolonged cardiac monitoring for atrial fibrillation detection after stroke: in search for the elusive sweet spot. Neurology. (2022). doi: 10.1212/wnl.0000000000200333 [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Fridman S, Jimenez-Ruiz A, Vargas-Gonzalez JC, Sposato LA. Differences between Atrial fibrillation detected before and after stroke and TIA: a systematic review and meta-analysis. Cerebrovasc Dis. (2022) 51:152–7. doi: 10.1159/000520101

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Krawczyk M, Fridman S, Cheng Y, Fang J, Saposnik G, Sposato LA. Atrial fibrillation diagnosed after stroke and dementia risk: cohort study of first-ever ischaemic stroke patients aged 65 or older. Europace. (2019) 21:1793–801. doi: 10.1093/europace/euz237

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Bhatla A, Borovskiy Y, Katz R, Hyman MC, Patel PJ, Arkles J, et al. Stroke, timing of atrial fibrillation diagnosis, and risk of death. Neurology. (2021) 96:e1655–62. doi: 10.1212/wnl.0000000000011633

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Benjamin EJ, Go AS, Desvigne-Nickens P, Anderson CD, Casadei B, Chen LY, et al. Research priorities in atrial fibrillation screening: a report from a national heart, lung, and blood institute virtual workshop. Circulation. (2021) 143:372–88. doi: 10.1161/circulationaha.120.047633

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Proietti M, Lane DA, Boriani G, Lip GYH. Stroke prevention, evaluation of bleeding risk and anticoagulant treatment management in atrial fibrillation contemporary international guidelines. Can J Cardiol. (2019) 35:619–33. doi: 10.1016/j.cjca.2019.02.009

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Aguilar M, Macle L, Deyell MW, Yao R, Hawkins N, Khairy P, et al. The influence of monitoring strategy on assessment of ablation success and post-ablation atrial fibrillation burden assessment: implications for practice and clinical trial design. Circulation. (2021) 145:21–30. doi: 10.1161/circulationaha.121.056109

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Bang OY, Ovbiagele B, Kim JS. Evaluation of cryptogenic stroke with advanced diagnostic techniques. Stroke. (2014) 45:1186–94. doi: 10.1161/strokeaha.113.003720

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Tsivgoulis G, Triantafyllou S, Palaiodimou L, Grory BM, Deftereos S, Köhrmann M, et al. Prolonged cardiac monitoring and stroke recurrence: a meta-analysis. Neurology. (2022). doi: 10.1212/wnl.0000000000200227 [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Hindricks G, Potpara T, Dagres N, Arbelo E, Bax JJ, Blomström-Lundqvist C, et al. 2020 ESC guidelines for the diagnosis and management of atrial fibrillation developed in collaboration with the European association of cardio-thoracic surgery (EACTS)the task force for the diagnosis and management of atrial fibrillation of the European society of cardiology (ESC) developed with the special contribution of the European heart rhythm association (EHRA) of the ESC. Eur Heart J. (2020) 42:ehaa612. doi: 10.1093/eurheartj/ehaa612

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: atrial fibrillation, external validation, ischemic stroke, prediction, risk score

Citation: Hsieh C-Y, Kao H-M, Sung K-L, Sposato LA, Sung S-F and Lin S-J (2022) Validation of Risk Scores for Predicting Atrial Fibrillation Detected After Stroke Based on an Electronic Medical Record Algorithm: A Registry-Claims-Electronic Medical Record Linked Data Study. Front. Cardiovasc. Med. 9:888240. doi: 10.3389/fcvm.2022.888240

Received: 02 March 2022; Accepted: 11 April 2022;
Published: 29 April 2022.

Edited by:

Charles Guenancia, University Hospital Dijon, France

Reviewed by:

Karim Benali, Saint-Etienne University Hospital Center, France
Hongwen Fei, Guangdong Cardiovascular Institute, Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences, China
Wengen Zhu, Sun Yat-sen University, China

Copyright © 2022 Hsieh, Kao, Sung, Sposato, Sung and Lin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Sheng-Feng Sung, sfusng@cych.org.tw; richard.sfsung@gmail.com

These authors have contributed equally to this work and share first authorship

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.