Predicting 30-Day Readmission for Stroke Using Machine Learning Algorithms: A Prospective Cohort Study

Background: Machine learning algorithms for predicting 30-day stroke readmission are rarely discussed. The aims of this study were to identify significant predictors of 30-day readmission after stroke and to compare prediction accuracy and area under the receiver operating characteristic (AUROC) curve in five models: artificial neural network (ANN), K nearest neighbor (KNN), random forest (RF), support vector machine (SVM), naive Bayes classifier (NBC), and Cox regression (COX) models.

Methods: The subjects of this prospective cohort study were 1,476 patients with a history of admission for stroke to one of six hospitals between March, 2014, and September, 2019. A training dataset (n = 1,033) was used for model development, and a testing dataset (n = 443) was used for internal validation. Another 167 patients with stroke recruited from October, to December, 2019, were enrolled in the dataset for external validation. A feature importance analysis was also performed to identify the significance of the selected input variables.

Results: For predicting 30-day readmission after stroke, the ANN model had significantly (P < 0.001) higher performance indices compared to the other models. According to the ANN model results, the best predictor of 30-day readmission was PAC followed by nasogastric tube insertion and stroke type (P < 0.05). Using a machine learning ANN model to obtain an accurate estimate of 30-day readmission for stroke and to identify risk factors may improve the precision and efficacy of management for these patients.

Conclusion: Using a machine-learning ANN model to obtain an accurate estimate of 30-day readmission for stroke and to identify risk factors may improve the precision and efficacy of management for these patients. For stroke patients who are candidates for PAC rehabilitation, these predictors have practical applications in educating patients in the expected course of recovery and health outcomes.

Introduction

Globally, stroke is not only the second leading cause of death, but also the disease with the second largest healthcare burden as estimated in disability-adjusted life-years (1). Previous studies have estimated that as many as 21% of stroke patients are readmitted within 30 days and have found that unplanned Medicare readmission in 2004 estimated in excess of $17 billion in costs (2–4). Furthermore, the mortality rate for 30-day readmission after stroke is more than 2.5 times greater than index admissions and highest among those readmitted for recurrent stroke (2). Additionally, one current study found that ~25.4% of the venous thromboembolism (VTE)-related hospital readmissions occurred within the first 30 days of discharge and they also estimated the mean cost for a hospital readmission with a primary diagnosis of VTE was $18,681; for readmissions with a primary diagnosis of deep vein thrombosis and pulmonary embolism, mean costs were $14,719 and $23,305, respectively (5). Reducing readmission rates among hospitals has become a goal of national healthcare reform.

This prospective study evaluated the use of machine learning algorithms for predicting 30-day readmission after stroke, univariate analysis and feature importance analysis. This study presented a novel opportunity to evaluate the use of post-acute care (PAC) history, demographic characteristics, clinical characteristics, and functional status outcomes as predictors of 30-day readmission in patients with stroke. The results of this study could be used to improve precision and efficacy in managing these patients. These results not only validate the use of similar prediction models for clinical practice in other countries, they also indicate that both PAC and analysis of functional status outcomes should be routinely be integrated in the care for stroke patients.

Although prior works to stratify risk of stroke outcomes have utilized basic statistical models, such as logistic regression been proposed recently, models for predicting readmission have had three major shortcomings. Firstly, recently proposed machine learning models have shown superior area under the receiver operating characteristic (AUROC) curve compared to conventional regression models in predicting 30-day readmission (range: 0.729–0.834 vs. 0.714–0.828, respectively) (6–8). Secondly, proposed forecasting models require use of health insurance claims data, which would not be available in a real-time clinical setting (9). Thirdly, previous studies predicted the risk of readmission do not comprehensively consider baseline patient characteristics, including post-acute care (PAC) history, demographic characteristics, comorbidities, and functional status score (10–12). However, literature on their use for predicting 30-day readmission for stroke is relatively sparse. The current studies regarding to 30-day readmission for patients with cerebrovascular diseases by using machine learning are summarized in Table 1 (6–9, 13–15).

TABLE 1

Table 1. The studies in predicting 30-day readmission for patients by using machine learning.

To reduce 30-day readmission after stroke and subsequent mortality, identifying factors that predict readmission is crucial. Determining the risk factors for 30-day readmission may be useful for developing policies for preventing readmission after stroke. Therefore, the aims of this study were to compare forecasting accuracy in the artificial neural network (ANN), K nearest neighbor (KNN), random forest (RF), support vector machine (SVM), naive Bayes classifier (NBC) and Cox regression (COX) models and to explore significant predictors of readmission within 30 days after stroke. The key contributions of this study can be summarized as follows:

• Advances in artificial intelligence have been applied in clinical practice. However, machine learning algorithms have not been used to predict 30-day readmission for patients with stroke mainly because of the high complexity of prediction algorithms relative to diagnostic algorithms.

• The proposed machine learning algorithms exhibit strong potential for use in predicting readmission within 30 days after stroke.

• A feature importance analysis was also performed to determine the significance of the selected input variables.

Materials and Methods

Study Design and Patients

The subjects of this prospective cohort study were 1,476 patients with a record of an ICD-9-CM (433.01, 433.10, 433.11, 433.21, 433.31, 433.81, 433.91, 434.00, 434.01, 434.11, 434.91 and 436 for ischemic stroke; 430 and 431 for hemorrhagic stroke), ICD-10 (I60–I62 were used to identify hemorrhagic stroke; I63 was used for ischemic stroke), and a history of admission to the PAC ward at one of four hospitals (three regional hospitals and one district hospital) or to a traditional non-PAC ward at one of two medical centers in south Taiwan between March, 2014, and September, 2019. The enrollment criteria were patients hospitalized for their first-ever stroke who were examined within 30 days with computed tomography (CT) or magnetic resonance imaging (MRI) and a Modified Rankin Scale (MRS) score of 2 to 4. Scores for the MRS range from 0 to 6, and a high MRS score indicates a high severity of disability. Patients were excluded if PAC beds were unavailable at the participating hospitals or if they had been transferred to PAC wards at other hospitals. In this scale, absence of symptoms is scored as 0. No significant disability, slight disability moderate disability moderately severe disability, and severe disability is scored as 1, 2, 3, 4 and 5, respectively (16). Another 167 stroke patients were recruited from October to December, 2019 (Figure 1). Figure 2 also depicts the conceptual framework of the proposed method for predicting readmission within 30 days after stroke. The study protocol was approved by the institutional review board at Kaohsiung Medical University Hospital (KMUH-IRB-20140308), and written informed consent was obtained from each participant.

FIGURE 1

Figure 1. Flowchart of the study.

FIGURE 2

Figure 2. Conceptual framework of the proposed method for predicting readmission within 30 days after stroke.

Instruments and Potential Predictors

Functional disability was measured using the 10-item Barthel Index (BI) (17). The BI measures functional disability in terms of inability to perform certain daily life activities (e.g., dressing, performing self-care, and walking up and down stairs). A BI score of 10 indicates complete independence. In stroke patients who had dysphagia, functional oral intake was assessed with the Functional Oral Intake Scale (FOIS) (18), in which swallowing function is classified on a scale from 1 (nil by mouth) to 7 (total oral diet without restriction). Cognitive status was quantitatively assessed with the Mini-mental State Examination (MMSE) (19). The MMSE includes tests for orientation, memory, attention, calculation, language, and construction functions where higher scores indicate better functional status (total score range, 0–30). The Instrumental Activities of Daily Living (IADL) scale is most useful for assessing current function and improvement or deterioration in function over time (20). When the IADL scale is administered in women, all eight domains for function are scored. In men, the domains of food preparation, housekeeping, and laundering are not scored. The EuroQoL Quality of Life Scale (EQ-5D-3L) measures the total health state of the subject based on a self-assessment of 5 items: mobility, self-care, usual activities, pain or discomfort, and anxiety or depression (21). Each EQ-5D-3L item is scored as 1 (no problem), 2 (some problem), or 3 (extreme problem). The 14-item Berg Balance Scale (BBS) is used to measure functional balance (22). Each item is rated from 0 (poor) to 4 (good), and the maximum score is 56. The Chinese versions of all instruments used in this study have been validated and used extensively in both clinical practice and research (17, 23).

A research assistant collected the following data from medical records after index discharge: PAC program (PAC group or non-PAC group), patient attributes (age, gender, education, and BMI), clinical attributes [stroke type, NG tube, Foley catheter, hypertension, diabetes mellitus (DM), hyperlipidemia, atrial fibrillation, previous stroke, acute care LOS, and rehabilitation ward LOS]. In multivariate analysis, the potential predictors were the independent variables, and 30-day readmission was the dependent variable.

Machine Learning Algorithms

Machine learning algorithms are effective tools for identifying and classifying readmission within 30 days after discharge in patients with stroke. Previous studies have successfully used machine learning to classify stroke according to characteristics such as cardiac source and gait in various scenarios (24, 25). In the present study, machine learning algorithms used to predict 30-day readmission in patients with stroke included ANN, KNN, RF, SVM, NBC and COX models.

Statistical Analysis

The unit of analysis in this study was the individual patient with stroke. Statistical analysis was performed in the following steps. In the first step, the statistical significance of continuous variables was tested by one-way analysis of variance, and that of categorical variables was tested by Fisher exact analysis. Univariate analyses were performed to identify significant predictors (P < 0.05). In the second step, data for the study cohort of 1,476 subjects were randomly divided into two datasets: a training dataset containing data for 1,033 subjects (70%), which was used for model development, and a testing dataset containing data for 443 subjects (30%), which was used for internal validation. A validation dataset containing data for another 167 patients enrolled after September, 2019, was used for external validation. To identify the optimal hyper-parameters for the machine learning algorithms, we applied Bayesian optimization using the expected improvement as the acquisition function (26). To perform the hyperband method of optimization and to test different combinations of hyper-parameters, we used Optuna version 2.10.0 (27). A total of 1,000 trials were conducted, and the parameters with the greatest area under the receiver operating characteristic curve were saved. Additionally, since data used for model fitting tended to overestimate model performance on unseen subjects, we coupled 10-fold cross-validation (28) with the logistic loss metric to measure the generalizability of the model to unseen subjects during model selection. A total of six machine-learning classifiers were constructed in the training dataset and tested in the validating dataset. A confusion matrix is used to describe and visualize the performance of the machine learning algorithm classifier and also to provide insight on what the model misclassifies. In the present study, the performance of the machine learning algorithms for the best classification task was evaluated in terms of confusion matrix-based performance measuring metrics including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy. In order to evaluate and select the most accurate machine learning algorithms, we used a confusion matrix and calculated the percentage of sensitivity, specificity, and accuracy of each forecasting model. In addition, the performance of the machine learning algorithms in the present study was also evaluated by the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUROC). The independent variables fitted to the forecasting models were significant predictors of 30-day readmission, and the dependent variable was 30-day readmission. After model training, model outputs were collected for each testing dataset. In the third step, bootstrapping, a machine learning technique, which involves taking random samples from the dataset with re-selection of 1,000 resamples was used to compare different machine learning algorithms employing the performance indices and the 95% confidence intervals. We used paired t-test to identify performance indices that significantly differed between the two models.

In the fourth and final step, feature selection method was calculated by using an algorithm to obtain an importance score for each potential predictor in the dataset (29). Feature importance analysis provides information about how each feature contributes to model prediction accuracy. The final weight of each feature is calculated by averaging the decrease in model accuracy after random permutation of the feature values within a testing set. Permutation of an important feature should decrease the score whereas permutation of a feature that is not very important to model prediction accuracy should increase the score. To obtain robust results with our small dataset, the train-test split was performed with a repeated stratified K fold cross validation. This technique has two advantages: first, it is model-agnostic; second, it can be performed repeatedly with different feature permutations. All statistical analyses were performed using the STATISTICA 13.0 software package (StatSoft, Inc., Tulsa, OK, USA). All statistical tests were two-sided; a P-value < 0.05 was considered statistically significant.

Results

Study Characteristics

Table 1 shows that 1,283 patients (86.9%) joined the per-diem PAC program and the remaining patients selected the fee-for-service non-PAC program. The patients with stroke had a mean age of 65.5 years (standard deviation, SD 13.0 years), and most (62.5%) patients were male. During the study period, 120 patients with stroke were readmitted within 30 days. In univariate analysis, PAC program, age, gender, education, body mass index (BMI), stroke type, nasogastric (NG) tube, Foley, hypertension, diabetes mellitus (DM), hyperlipidemia, atrial fibrillation, previous stroke, acute care length of stay (LOS), rehabilitation LOS and functional status score before rehabilitation were significantly associated with 30-day readmission (P < 0.05). These significant predictors were included in the forecasting models (Table 2).

TABLE 2

Table 2. Baseline characteristics of the study population (N = 1,476).

Comparison of Forecasting Models

Significant predictors of 30-day readmission did not significantly differ between the training and testing datasets; therefore, samples were compared between the training and testing datasets to increase reliability of the validation results (Table 3). We used grid search to find the best hyperparameters for the neural networks. We searched for the following hyperparameters: the number of hidden layers (in the range of 1–6), the number of hidden neurons in each layer (in the range of 1–512), activation functions (“relu,” “logistic sigmoid”), and learning rate (in the range of 0.01–0.001). We used adam optimizer, constant learning rate, and the regularization rate of alpha = 0.01. The SVM model was configured with linear kernel, and regularization parameter C = 1.0. The RF model is an ensemble learning method combined of multiple decision tree predictors that are trained based on random data samples and feature subsets. We configured the RF algorithm with two trees in the forest. Hyperparameter optimization was then performed to improve the performance of the compact model, and the machine learning algorithms with the greatest AUROC values in 1,000 trials were obtained. Table 4 lists the final hyperparameter settings. The data in Table 5 indicate that the ANN model compared to KNN, RF, SVM, NBC, and COX models had significantly (P < 0.001) higher sensitivity, specificity, PPV, NPV, accuracy, and AUC values. Similar results also were shown in dataset for testing simultaneously. The receiver operating characteristic (ROC) curve results in Figure 3 show that the ANN model had significantly higher ROC values compared to other forecasting models (P < 0.001).

TABLE 3

Table 3. Univariate analysis of selected risk factors for 30-day readmission in patients with stroke (N = 1,476).

TABLE 4

Table 4. Hyper-parameters and final settings in all machine learning algorithms.

TABLE 5

Table 5. Comparison of 1,000 pairs of forecasting models for predicting 30-day readmission in patients with stroke (N = 1,476).

FIGURE 3

Figure 3. Performance indices of forecasting models used to predict 30-day readmission in patients with stroke when using (A) training dataset, (B) testing dataset. The box plot shows the median (centers) and interquartile range (borders). In analyses of accuracy and AUROC, the ANN model had significantly higher values compared to other forecasting models (P < 0.001). AUROC, area under the receiver operating characteristics; ANN, artificial neural network.

Significant Predictors in the ANN Model

Figure 4 shows the feature importance analysis results for the ANN model. The VSR value for predicting 30-day readmission in stroke patients was highest for PAC (permutation importance = 0.761) followed by NG tube (0.552), stroke type (0.448), BI score before rehabilitation (0.423), IADL score before rehabilitation (0.418), MMSE score before rehabilitation (0.409), BBS score before rehabilitation (0.408), FOIS score before rehabilitation (0.404), EQ5D score before rehabilitation (0.401), and others.

FIGURE 4

Figure 4. A permutation importance analysis of artificial neural network model in predicting 30-day readmission in patients with stroke. BI, Barthel Index; IADL, Instrumental Activities of Daily Living; MMSE, Mini-Mental State Examination; BBS, Berg Balance Scale; FOIS, Functional Oral Intake Scale; EQ-5D, EuroQoL Quality of Life Scale.

Sensitivity Analysis

Next, the validating dataset of 167 subjects was used to compare the predictive accuracy of the models. Table 6 also compares the performance indices obtained in external validation of the ANN, KNN, RF, SVM, NBC and COX models. For predicting 30-day readmission, the ANN model consistently achieved significantly higher performance indices (P < 0.001).

TABLE 6

Table 6. Comparative performance indices of forecasting models when using 167 new validating datasets to predict 30-day readmission in patients with stroke.

Discussion

Accuracy in predicting 30-day readmission in patients with stroke was compared among five forecasting models. For a given set of clinical inputs, the ANN model clearly had superior forecasting accuracy compared to the other four. Notably, our prospective study collected longitudinal data from six different medical institutions, which provided a real-world depiction of current treatment for patients with stroke. In contrast, previous works have used data from a single medical center (10–13). Moreover, using registry data obtained from six hospitals mitigated the potential for referral bias or bias caused by analyzing the practices of a single physician or a single institution (30, 31).

Recent works have demonstrated the superior performance of machine learning-based models for predicting stroke outcomes (24, 25). One advantage of using an ANN model is that it enables appropriate and accurate processing of inputs that are incomplete or inputs that introduce noise (9, 32). Another advantage of ANN models, whether linear or non-linear, is their good performance in/effectiveness for analyzing large-scale medical databases constructed using data that are highly correlated but not normally distributed. The high robustness of the ANN model has been demonstrated in many clinical applications, particularly predicting prognosis in various diseases (32). In performance comparisons of the five models in this study, expanding the number of potential predictors apparently improved the performance of the ANN model in systematic analysis of outcome in various diseases.

Our current results indicate that ANN models can use clinical outcome data for predicting 30-day readmission after stroke. Prospective prediction performance and cross-validation performance were adequate when subjects were familiar with the task and when information from the previous test session was made available. However, larger scale studies are still needed to validate this approach.

A permutation importance analyses of the weights of significant predictors of 30-day readmission for stroke revealed that the best predictor was PAC. This finding is consistent with earlier reports that, in comparisons of independent predictors, PAC is the best predictor of stroke outcome, including overall treatment cost, functional status after stroke, and duration of hospital stay before transfer to rehabilitative ward (30, 33). In a quasi-experimental study of stroke patients, Wang et al. (30) investigated the longitudinal impact of PAC on functional status. The authors concluded that multidisciplinary rehabilitative PAC delivered on a per-diem basis substantially improved functional status compared to standard rehabilitation. Another study performed in a nationwide stroke cohort compared mortality and numerous functional domains between a PAC group and a well-matched non-PAC group (34). The PAC group had significantly lower 90-day hospital readmissions and stroke-related readmissions compared to the non-PAC group.

Dennis et al. (35) reported that, compared to nasogastric feeding, percutaneous endoscopic gastrostomy was associated with higher risk of death or poorer outcomes at 6 months after stroke. However, Ho et al. (36) noted that prolonged (i.e., longer than 2 weeks) nasogastric tube feeding was significantly associated with pneumonia and mortality. In the current study, NG tube insertion before rehabilitation was significantly associated with 30-day readmission (P < 0.001). During the study period, no patient with stroke required NG tube insertion after rehabilitation.

Compared to other stroke types, hemorrhagic stroke is reportedly associated with higher severity and with higher overall mortality in the first 3 months after stroke (37, 38). The current study further revealed that hemorrhagic stroke has a higher 30-day readmission rate for ischemic stroke.

This prospective observational cohort study of patients with stroke in Taiwan analyzed data from patients treated at six healthcare institutions. The predictive accuracy of the ANN model developed in this study outperformed the other four models in identifying predictors of 30-day readmission. Three implications of this study are noted. First, the proposed ANN model may be useful for guiding the clinical care of patients with stroke. Second, healthcare administrators and managers at medical institutions should facilitate prompt and appropriate PAC for patients with stroke. Third, the Taiwan National Health Insurance Administration should include PAC in its guidelines for clinical treatment of stroke in order to achieve a broad nationwide improvement in care for these patients. However, further studies are needed to confirm the clinical relevance of the proposed ANN model in terms of its efficacy in predicting prognosis and optimizing medical management for patients with stoke.

For further validation of the significant association observed between PAC and 30-day readmission for stroke, Table 7 compares six relevant studies performed in the United States or Taiwan (39–44). The six studies shared the following features: (1) a relatively large sample size, (2) a mean age of 65 years or more, (3) use of statewide or national datasets, and, most importantly, (4) investigation of 30-day readmission in patients with stroke. As in these previous works, out study demonstrated a significantly lower 30-day stroke readmission rate in a multidisciplinary PAC group compared to a non-PAC group (P < 0.001).

TABLE 7

Table 7. Reported associations between post-acute care (PAC) for stroke and 30-day readmission.

This study has several limitations inherent in a large database analysis. First, the validity of the comparisons in the study is limited by the exclusion of complications associated with stroke rehabilitation outcomes. Second, the analysis was limited to 30-day readmission, which reduces the subset of patients with stroke in which the ANN model is clinically applicable. Third, imbalance between positive and negative outcomes, i.e., class imbalance, is a common problem in analysis of medical data and has not been satisfactorily addressed (45, 46). Further studies are needed to investigate the use of ensemble algorithm for solving the class imbalance problem. Additionally, whether the timing or duration of the stroke treatment is a relevant prognostic predictor of readmission deserves further study. Nevertheless, the results can still be considered valid given the robustness and statistical significance of the results.

Conclusions

Based on the comparison results in this study, we conclude that the ANN model is superior to the other forecasting models in terms of accuracy in predicting 30-day readmission for stroke after a hospital discharge. The ANN model outperformed the other models in terms of both accuracy and AUROC curve. Using a machine-learning ANN model to obtain an accurate estimate of 30-day readmission for stroke and to identify risk factors may improve the precision and efficacy of management for these patients. Predictors of stroke can be discussed when educating PAC candidates in the expected course of recovery and health outcomes. Although the practical applicability of database studies such as this have been convincingly demonstrated in the literature, future studies can expand the range of clinical variables included in the analysis, which could obtain additional results and potentially improve prediction accuracy. Such data could be vital for developing, promoting, and improving health policies for treating patients with stroke.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The study protocol was approved by the Institutional Review Board at Kaohsiung Medical University Hospital (KMUH-IRB-20140308) and written informed consent was obtained from each participant.

Author Contributions

Y-CC and H-YS: conceptualization, data curation, formal analysis, investigation, methodology, resources, software, validation, visualization, writing—original draft, and writing—review and editing. J-HC, Y-JY, S-JL, H-FL, C-HL, H-HH, K-WH, and S-CY: data curation, formal analysis, investigation, methodology, resources, software, validation, and visualization. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by funding from the NSYSU-KMU JOINT RESEARCH PROJECT (NSYSU-KMU 109-P001 and 110-P017), NPUST-KMU JOINT RESEARCH PROJECT (NPUST-KMU 109-P010, 110-P001, and 111-P010), and the Ministry of Science and Technology (MOST 104-2410-H-037-006-SS2, MOST 106-2410-B-037-076, and MOST 108-2410-H-037-006-SS3) in Taiwan. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Ottenbacher KJ, Karmarkar A, Graham JE, Kuo YF, Deutsch A, Reistetter TA, et al. Thirty-day hospital readmission following discharge from postacute rehabilitation in fee-for-service medicare patients. JAMA. (2014) 311:604–14. doi: 10.1001/jama.2014.8

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Nouh AM, McCormick L, Modak J, Fortunato G, Staff I. High mortality among 30-day readmission after stroke: predictors and etiologies of readmission. Front Neurol. (2017) 8:632. doi: 10.3389/fneur.2017.00632

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Fehnel CR, Lee Y, Wendell LC, Thompson BB, Potter NS, Mor V. Post-acute care data for predicting readmission after ischemic stroke: a nationwide cohort analysis using the minimum data set. J Am Heart Assoc. (2015) 4:e002145. doi: 10.1161/JAHA.115.002145

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Liotta EM, Singh M, Kosteva AR, Beaumont JL, Guth JC, Bauer RM, et al. Predictors of 30 day readmission after intracerebral hemorrhage: a single-center approach for identifying potentially modifiable associations with readmission. Crit Care Med. (2013) 41:2762–9. doi: 10.1097/CCM.0b013e318298a10f

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Amin A, Deitelzweig S, Bucior I, Lin J, Lingohr-Smith M, Menges B, et al. Frequency of hospital readmissions for venous thromboembolism and associated hospital costs and length of stay among acute medically ill patients in the US. J Med Econ. (2019) 22:1119–25. doi: 10.1080/13696998.2019.1618862

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Lineback CM, Garg R, Oh E, Naidech AM, Holl JL, Prabhakaran S. Prediction of 30-day readmission after stroke using machine learning and natural language processing. Front Neurol. (2021) 12:649521. doi: 10.3389/fneur.2021.649521

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Darabi N, Hosseinichimeh N, Noto A, Zand R, Abedi V. Machine learning-enabled 30-day readmission model for stroke patients. Front Neurol. (2021) 12:638267. doi: 10.3389/fneur.2021.638267

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Xu Y, Yang X, Huang H, Peng C, Ge Y, Wu H, et al. Extreme gradient boosting model has a better performance in predicting the risk of 90-day readmissions in patients with ischaemic stroke. J Stroke Cerebrovasc Dis. (2019) 28:104441. doi: 10.1016/j.jstrokecerebrovasdis.2019.104441

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Sarajlic P, Simonsson M, Jernberg T, Bäck M, Hofmann R. Incidence, associated outcomes, and predictors of upper gastrointestinal bleeding following acute myocardial infarction: A SWEDEHEART-based nationwide cohort study. Eur Heart J Cardiovasc Pharmacother. (2021) pvab059. doi: 10.1093/ehjcvp/pvab059

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Zhong W, Geng N, Wang P, Li Z, Cao L. Prevalence, causes and risk factors of hospital readmissions after acute stroke and transient ischemic attack: a systematic review and meta-analysis. Neurol Sci. (2016) 37:1195–202. doi: 10.1007/s10072-016-2570-5

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Wen T, Liu B, Wan X, Zhang X, Zhang J, Zhou X, et al. Risk factors associated with 31-day unplanned readmission in 50,912 discharged patients after stroke in China. BMC Neurol. (2018) 18:218. doi: 10.1186/s12883-018-1209-y

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lin HJ, Chang WL, Tseng MC. Readmission after stroke in a hospital-based registry: risk, etiologies, and risk factors. Neurology. (2011) 76:438–43. doi: 10.1212/WNL.0b013e31820a0cd8

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Sharma V, Kulkarni V, McAlister F, Eurich D, Keshwani S, Simpson SH, et al. Predicting 30-day readmissions in patients with heart failure using administrative data: a machine learning approach. J Card Fail. (2022) 28:710–22. doi: 10.1016/j.cardfail.2021.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wang Z, Chen X, Tan X, Yang L, Kannapur K, Vincent JV, et al. Using deep learning to identify high-risk patients with heart failure with reduced ejection fraction. J Health Econ Outcomes Res. (2021) 8:6–13. doi: 10.36469/jheor.2021.25753

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Amritphale A, Chatterjee R, Chatterjee S, Amritphale N, Rahnavard A, Awan GM, et al. Predictors of 30-day unplanned readmission after carotid artery stenting using artificial intelligence. Adv Ther. (2021) 38:2954–72. doi: 10.1007/s12325-021-01709-7

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Quinn TJ, McArthur K, Dawson J, Walters MR, Lees KR. Reliability of structured modified Rankin scale assessment. Stroke. (2010) 41:e602. doi: 10.1161/STROKEAHA.110.590547

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Leung SO, Chan CC, Shah S. Development of a Chinese version of the Modified Barthel Index– validity and reliability. Clin Rehabil. (2007) 21:912–22. doi: 10.1177/0269215507077286

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Crary MA, Mann GD, Groher ME. Initial psychometric assessment of a functional oral intake scale for dysphagia in stroke patients. Arch Phys Med Rehabil. (2005) 86:1516–20. doi: 10.1016/j.apmr.2004.11.049

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Agrell B, Dehlin O. Mini mental state examination in geriatric stroke patients. Validity, differences between subgroups of patients, and relationships to somatic and mental variables. Aging. (2000) 12:439–44. doi: 10.1007/BF03339874

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Vittengl JR, White CN, McGovern RJ, Morton BJ. Comparative validity of seven scoring systems for the instrumental activities of daily living scale in rural elders. Aging Ment Health. (2006) 10:40–7. doi: 10.1080/13607860500307944

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol group. Ann Med. (2001) 33:337–43. doi: 10.3109/07853890109002087

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chou CY, Chien CW, Hsueh IP, Sheu CF, Wang CH, Hsieh CL. Developing a short form of the Berg Balance Scale for people with stroke. Phys Ther. (2006) 86:195–204. doi: 10.1093/ptj/86.2.195

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Wang CY, Miyoshi S, Chen CH, Lee KC, Chang LC, Chung JH, et al. Walking ability and functional status after post-acute care for stroke rehabilitation in different age groups: a prospective study based on propensity score matching. Aging. (2020) 12:10704–14. doi: 10.18632/aging.103288

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Park SJ, Hussain I, Hong S, Kim D, Park H, Benjamin HCM. Real-time gait monitoring system for consumer stroke prediction service. 2020 IEEE Int Conf Consum Electron. (2020) 2020:1–4. doi: 10.1109/ICCE46568.2020.9043098

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hussain I, Park SJ. Big-ECG: Cardiographic predictive cyber-physical system for stroke management. IEEE Access. (2021) 9:123146–64. doi: 10.1109/ACCESS.2021.3109806

CrossRef Full Text | Google Scholar

26. Kamogashira T, Fujimoto C, Kinoshita M, Kikkawa Y, Yamasoba T, Iwasaki S. Prediction of vestibular dysfunction by applying machine learning algorithms to postural instability. Front Neurol. (2020) 11:7. doi: 10.3389/fneur.2020.00007

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Li L, Jamieson K, Desalvo G, Rostamizadeh A, Talwalkar A. Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res. (2017) 18:6765–816. doi: 10.48550/arXiv.1603.06560

CrossRef Full Text | Google Scholar

28. Wong TT. Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recogn. (2015) 48:2839–46. doi: 10.1016/j.patcog.2015.03.009

CrossRef Full Text | Google Scholar

29. Altmann A, Toloşi L, Sander O, Lengauer T. Permutation importance: a corrected feature importance measure. Bioinformatics. (2010) 26:1340–7. doi: 10.1093/bioinformatics/btq134

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Wang CY, Hsien HH, Hung KW, Lin HF, Chiou HY, Yeh SCJ, et al. Multidiscipline stroke post-acute care transfer system: propensity-score-based comparison of functional status. J Clin Med. (2019) 8:1233. doi: 10.3390/jcm8081233

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Jahan R, Saver JL, Schwamm LH, Fonarow GC, Liang L, Matsouaka RA, et al. Association between time to treatment with endovascular reperfusion therapy and outcomes in patients with acute ischemic stroke treated in clinical practice. JAMA. (2019) 322:252–63. doi: 10.1001/jama.2019.8286

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Baxt WG. Application of artificial neural networks to clinical medicine. Lancet. (1995) 346:1135–8. doi: 10.1016/S0140-6736(95)91804-3

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wang CY, Chen YR, Hong JP, Chan CC, Chang LC, Shi HY. Rehabilitative post-acute care for stroke patients delivered by per-diem payment system in different hospitalization paths: a Taiwan pilot study. Int J Qual Health Care. (2017) 29:779–84. doi: 10.1093/intqhc/mzx102

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Peng LN, Lu WH, Liang CK, Chou MY, Chung CP, Tsai SL, et al. Functional outcomes, subsequent healthcare utilization, and mortality of stroke postacute care patients in Taiwan: a nationwide propensity score-matched study. J Am Med Dir Assoc. (2017) 18:990.e7–12. doi: 10.1016/j.jamda.2017.06.020

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Dennis MS, Lewis SC, Warlow C. FOOD Trial Collaboration. Effect of timing and method of enteral tube feeding for dysphagic stroke patients (FOOD): a multicentre randomised controlled trial. Lancet. (2005) 365:764–72. doi: 10.1016/S0140-6736(05)17983-5

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Ho CH, Lin WC, Hsu YF, Lee IH, Hung YC. One-year risk of pneumonia and mortality in patients with poststroke dysphagia: a nationwide population-based study. J Stroke Cerebrovasc Dis. (2018) 27:1311–7. doi: 10.1016/j.jstrokecerebrovasdis.2017.12.017

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Andersen KK, Olsen TS, Dehlendorff C, Lee IH, Hung YC. Hemorrhagic and ischemic strokes compared: stroke severity, mortality, and risk factors. Stroke. (2009) 40:2068–72. doi: 10.1161/STROKEAHA.108.540112

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Roth GA, Dwyer-Lindgren L, Bertozzi-Villa A, Stubbs RW, Morozoff C, Naghavi M, et al. Trends and patterns of geographic variation in cardiovascular mortality among us counties, 1980-2014. JAMA. (2017) 317:1976–92. doi: 10.1001/jama.2017.4150

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kim Y, Thirukumaran C, Temkin-Greener H, Hill E, Holloway R, Li Y. Institutional postacute care use may help reduce readmissions for ischemic stroke patients. Med Care. (2021) 59:736–42. doi: 10.1097/MLR.0000000000001568

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Kosar CM, Loomer L, Ferdows NB, Trivedi AN, Panagiotou OA, Rahman M. Assessment of rural-urban differences in postacute care utilization and outcomes among older US adults. JAMA Netw Open. (2020) 3:e1918738. doi: 10.1001/jamanetworkopen.2019.18738

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Raman N, Al-Robaidi K, Jadhav A, Thirumala PD. Perioperative stroke and readmissions rates in noncardiac non-neurologic surgery. J Stroke Cerebrovasc Dis. (2020) 29:104792. doi: 10.1016/j.jstrokecerebrovasdis.2020.104792

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Li CY, Karmarkar A, Lin YL, Kuo YF, Ottenbacher KJ. Hospital readmissions reduction program and post-acute care: implications for service delivery and 30-day hospital readmission. J Am Med Dir Assoc. (2020) 21:1504–8. doi: 10.1016/j.jamda.2020.05.018

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Ramchand P, Thibault DP, Crispo JA, Levine J, Hurst R, Mullen MT, et al. Readmissions after mechanical thrombectomy for acute ischemic stroke in the united states: a nationwide analysis. J Stroke Cerebrovasc Dis. (2018) 27:2632–40. doi: 10.1016/j.jstrokecerebrovasdis.2018.05.035

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Hsieh CY, Tsao WC, Lin RT, Chao AC. Three years of the nationwide post-acute stroke care program in Taiwan. J Chin Med Assoc. (2018) 81:87–8. doi: 10.1016/j.jcma.2017.09.003

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Soto-Varela A, Rossi-Izquierdo M, Del-Río-Valeiras M, Faraldo-García A, Vaamonde-Sánchez-Andrade I, Lirola-Delgado A, et al. Modified timed up and go test for tendency to fall and balance assessment in elderly patients with gait instability. Front Neurol. (2020) 11:543. doi: 10.3389/fneur.2020.00543

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Wu J, Tang Y. Revisiting the immune balance theory: a neurological insight into the epidemic of COVID-19 and its alike. Front Neurol. (2020) 11:566680. doi: 10.3389/fneur.2020.566680

PubMed Abstract | CrossRef Full Text | Google Scholar