Predicting the behavioral intentions of hospice and palliative care providers from real-world data using supervised learning: A cross-sectional survey study

Background: Hospice and palliative care (HPC) aims to improve end-of-life quality and has received much more attention through the lens of an aging population in the midst of the coronavirus disease pandemic. However, several barriers remain in China due to a lack of professional HPC providers with positive behavioral intentions. Therefore, we conducted an original study introducing machine learning to explore individual behavioral intentions and detect factors of enablers of, and barriers to, excavating potential human resources and improving HPC accessibility.

Methods: A cross-sectional study was designed to investigate healthcare providers' behavioral intentions, knowledge, attitudes, and practices in hospice care (KAPHC) with an indigenized KAPHC scale. Binary Logistic Regression and Random Forest Classifier (RFC) were performed to model impacting and predict individual behavioral intentions.

Results: The RFC showed high sensitivity (accuracy = 0.75; F1 score = 0.84; recall = 0.94). Attitude could directly or indirectly improve work enthusiasm and is the most efficient approach to reveal behavioral intentions. Continuous practice could also improve individual confidence and willingness to provide HPC. In addition, scientific knowledge and related skills were the foundation of implementing HPC.

Conclusion: Individual behavioral intention is crucial for improving HPC accessibility, particularly at the initial stage. A well-trained RFC can help estimate individual behavioral intentions to organize a productive team and promote additional policies.

Introduction

An aging population is a great challenge for healthcare systems, especially for hospice and palliative care (HPC) aiming to improve end-of-life quality. The number of people aged 65 years or over was 703 million globally in 2019 and is projected to reach 1.5 billion by 2050 (1). Likewise, 176 million people (12.6%) aged 65 years or over in China, and the rising burden of non-communicable diseases (NCDs), poses severe challenges to the elder's quality of life (2) and the supply of HPC. Furthermore, living in environments affected by the coronavirus disease (COVID-19) pandemic and humanitarian crises are large-scale events that may pressure healthcare systems greatly. The World Health Organization (WHO) defines HPC as a crucial part of integrated care providing physical, mental, spiritual, and social care for older people and terminally ill patients by treating their discomfort, and is the basic skills of healthcare providers (3). Research shows that early delivery of HPC reduces unnecessary hospital admissions and services and optimizes patient health-related quality of life (4, 5). HPC should be available at all levels of care (6), but only 14% of patients currently have access to it worldwide. Policies for strengthening and expanding human resources are urgently needed to improve accessibility (7). The situation in China is more challenging due to the lack of professional HPC providers with positive behavioral intentions, poor public conception, and lack of knowledge regarding potential benefits (8). Also, targeted training is often limited, and HPC providers' knowledge and skills vary greatly, leading to unsatisfactory services (9). The WHO appeals to the local government and healthcare planners to estimate HPC demands and supplies using accurate data to combat the challenges (10). Therefore, there is an urgent need to evaluate the real-world situation of implementing HPC, particularly that of understanding the individual behavioral intentions that reflect whether healthcare providers are willing to engage in HPC. Exploration of the knowledge, attitude, and practice of hospice care (KAPHC) among doctors, nurses, and managers may yield useful information (11, 12), contribute to estimating individual behavioral intentions, and organize a productive service team to deliver HPC.

Existing studies mostly used traditional statistical methods to analyze the KAPHC among healthcare providers (13–15). There is abundant evidence suggesting that KAPHC is crucial for successful implementation of HPC (16–18). However, there is limited evidence about the use of combined real-world data and KAPHC to predict behavioral intentions. The main challenge of standardized surveys and many statistical models include limited data with unclear distribution. Thus, advanced machine learning and artificial intelligence enable the inclusion of large amounts of data based on correlational studies (19). Random Forest Classifier (RFC) may be more suitable for the analysis of behavioral intentions of HPC. One of the biggest advantages of RFC is that it can be used on data exhibiting highly unusual distribution by not making any distributional assumptions about underlying data structures. Additionally, it is a variation of bootstrap aggregating (bagging), which develops several hundred trees from the same dataset. The results are averaged from these trees, producing more accurate results to solve the problems of overfitting and being sensitive to small changes in the training data for a single Decision Tree (DT) (20). In addition, RFC allows the determination of feature importance measures for each parameter by measuring the effect of variable permutation on the model's accuracy (measured using out-of-bag error estimation) and node homogeneity (measured using the Gini index). In other words, it can tell each variable's importance in predicting the outcome (21). We applied the RFC to model impact and relate the prediction of individual behavioral intentions based on demographic parameters, career experience, and KAPHC to identify those that are willing to engage in HPC. RFC was introduced to predict behavioral intentions and review the model performance. In particular, this study aimed to detect correlative enablers and barriers in providing HPC.

Methods

Questionnaires

We conducted a cross-sectional study using the indigenized healthcare provider KAPHC scale, which proved to have good reliability (22). The scale mainly refers to the Palliative Care Quiz for Nursing by Ross et al. (23) and the Frommelt Attitudes Toward Care of the Dying Scale by Frommelt (24), Liu et al. (25) and Shimizu et al. (26). The three main components of the anonymous questionnaire were: demographic characteristics (gender and age group), career experience (location of the medical institution, type of medical institution), and measured data (score of knowledge, attitude, confidence, and practice).

Data collection

Shanghai had received continuous support to improve HPC research and practices and achieve universal coverage of HPC among community health service centers (CHC) by 2020 (27). Therefore, we organized all the 228 medical institutions that registered an HPC department or were willing to provide HPC in Shanghai, to investigate healthcare providers' KAPHC and behavioral intentions from November 1 to December 31, 2019. Overall, 3709 HPC providers with working experience in HPC were invited to fill in the questionnaire independently, including doctors, nurses, managers, medical technicians, and others.

Statistical methods

Descriptive statistics

The indicators were divided into two components: the target vector and the feature matrix. Behavioral intentions, a binary variable, were selected as the target vector, with the label (1/yes) suggesting positive behavioral intentions to work in HPC and the negative label (0/no) without behavioral intentions. The feature matrix includes demographic characteristics, career experience, and measure data of KAPHC. A higher score means better personal performance. Descriptive statistics for the feature matrix were based on full samples to analyze the situation of participation. Categorical variables were reported as frequency distribution and percentage, whereas continuous variables were reported as average values, standard deviations, median, and IQR (Inter Quartile Range). IBM SPSS Statistics for Windows, version 21.0 (IBM Corp., Armonk, NY, USA) software was used for the statistical analysis.

Logistic regression

Binary Logistic Regression (BLR) was performed to analyze the correlation between a dependent variable and independent variables to identify which variables can influence the willingness of healthcare providers to work in HPC. BLR could provide several key information points, namely, which explanatory variables show statistical significance with Wald's test, which variables are the risk or protective factors, and how important they are by odds ratio (OR). The goodness-of-fit is compared using the likelihood ratio test, the Hosmer–Lemeshow test, and Pearson's chi-squared test (28). BLR was used to select the relative potential features to control confounders and develop RFC. Moreover, the model fitting method was chosen as the stepwise way, and the probability for stepwise was reset such that entry was 0.1 and removal was 0.15 to avoid the type I error. Statistical analyses of the Logistic Regression model were performed using SPSS 21.0.

Random forest

Random Forest (RF) lowers the variance of a single regression tree by averaging the prediction of multiple DTs. The DT is a non-parametric supervised learning method used for classification and regression analysis. The DT predicts the value of a target variable by learning simple decision rules inferred from the data features, which can be seen as a piecewise constant approximation (29). Each DT consists of edge conjunctions on a single variable greater than or less than some value. Each tree node divides the data into two subsets, thus making each subset more homogeneous (Figure 1). The terminal node (leaf) of the tree that a certain data point falls into determines the predictions by the majority selection of the training data (30). RF is a meta-estimator that fits several DT classifiers on various subsamples of the dataset and uses averaging to improve the predictive accuracy and control overfitting (31). It is a method by which several DTs (tree-like graphs of decisions and their possible consequences, including event outcomes) are built from the variable set. They divide participants into similar subgroups using the most significant variables. The prediction is subsequently generated using a “voting” scheme across all DTs. Gradient descent boosting also picks variables across DTs that best predict correct outcomes in their training sets (32).

FIGURE 1

Figure 1. Part of decision tree. It reveals how decision tree works on the basis of Gini Impurity and other parameters. Random Forest is one of Ensemble Learnings developing from Decision Trees.

In our study, the variable types of latent factors include nominal variables and skewed continuous variables. Related studies have proven the applicability of RF in assembling learning algorithms and machine learning methods with the advantages of no restrictions on variable conditions and higher accuracy, sensitivity, and specificity. In addition, RF could be utilized to predict continuous variables and obtain predictions without obvious deviations (33). Therefore, RFC (one of typical RF models) is considered a suitable prediction method for the data in this study. RFC was performed using the program Python, version 3.9.0 (Python Software Foundation., Beaverton, OR, USA) with packages including Sklearn, Numpy, and Pandas.

Model development

We selected the potential features with statistical significance using BLR to develop our models. The data were normalized for having a zero mean and unit standard deviation, and randomly split into the 80% training dataset (n = 2,804) and the 20% test dataset (n = 701) (34). Considering the target vector was the binary variable, each feature was standardized between 0 and 1. Categorical features were encoded as the one-hot numeric array. This has created a binary column for each category and returned a sparse matrix or dense array (depending on the sparse parameter) (35). Model validation was performed using a 10-fold cross-validation approach on all the training data to avoid overfitting before RFC performance (36). Once preferable validation and reliability were confirmed, we trained the model on the complete training dataset, and the test dataset was used for the final validation. Meanwhile, RFC was compared with several widely used machine learning models to assess its advantage. Furthermore, the feature importance indicated the value of each feature in the RFC (37).

The predictive abilities of RFC were assessed using the following measures:

Accuracy: The proportion of the correctly classified samples. In multilabel classification, this function computes subset accuracy, meaning the set of labels predicted for a sample must match the corresponding set of labels in positive (38).

Precision: The ratio of “tp/(tp + fp)” where “tp” is the number of true positives and “fp” the number of false positives. The precision is intuitively the ability of the classifier not to label as positive a sample that is negative.

Recall: The ratio of “tp/(tp + fn)” where “tp” is the same as above and “fn” is the number of false negatives. The recall is intuitively the ability of the classifier to find all the positive samples.

F1 score: The weighted average of the precision and recall. The relative contribution of precision and recall to the F1 score are equal. F1 score = 2 ^* (precision ^* recall) / (precision + recall) (39).

AUC: The area under the Receiver Operating Characteristic curve (40).

Average Precision (AP): The Precision-Recall curve shows the tradeoff between precision and recalls for different thresholds. A high area under the curve represents high recall and high precision. The AP summarizes a Precision-Recall curve as the weighted mean of precisions achieved at each threshold (41).

Results

Participants characteristics

Overall, 3,505 valid questionnaires were gathered with an effective response rate of 94.5%. There were 63.4% participants aged 30–50 years, the majority (75.7%) were women, and 76.3% were married. Additionally, 68.8% held a bachelor's degree (or above), and 16.1% had religious beliefs. For career experience, approximately half (49.6%) worked downtown and the others (50.45%) in the countryside. The majority were employed in CHC (64.5%), and most participants were doctors (36.7%) and nurses (39.25%). The government funded most medical institutions (82.7%). Among all the participants with certain working experience in HPC, only half (56.1%) still work in HPC positions at the time of the investigation. Details are given in Table 1.

TABLE 1

Table 1. Characteristics of study participants (n = 3505).

The individual scores for attitude, practice, and requirement of training were more dispersed than those of knowledge and confidence. The average attitude was 91.7 points, and the standard deviation and IQR of attitude score were the highest among the five items. The average knowledge score was 8.9 points, with the lowest standard deviation and IQR. More details are reported in Table 2.

TABLE 2

Table 2. Measure data of studied participants (n = 3,505).

Feature selection

Figure 2 presents the result of the last step in BLR. The Wald test indicated that marital status (P = 0.031), location of medical institution (P = 0.050), type of medical institution (P < 0.001), fund or supply of medical institution (P = 0.014), occupation (P < 0.001), profession title (P = 0.021), “Working in hospice care now” (P < 0.001), attitude (P < 0.001), knowledge (P = 0.001), confidence (P < 0.001), and practice (P = 0.059) showed statistical significance; these factors were selected as potential features fitting in RFC analysis. Meanwhile, it should be noted that KAPHC showed a positive impact on behavioral intentions, especially knowledge (OR = 1.054; 95% CI: 1.021–1.089), which is higher than others. Meanwhile, profession title, marital status 2 (Divorced or widow), medical institution 2 (CHCs), and occupation 2 (doctor) showed a negative impact on behavioral intentions, especially the medical institution of CHCs (OR = 0.312; 95% CI: 0.185–0.526).

FIGURE 2

Figure 2. Forest plot of binary logistic regression. Only the variables with statistical significance were displayed in the plot.

Cross-validation

Overall, most measures were over 0.75. The recall performance (average of 0.92) was the best, followed by AP and F1-score of about 0.85. Additionally, the average accuracy, precision, and AUC values were between 0.75 and 0.80. Details are given in Table 3.

TABLE 3

Table 3. Result of cross-validation on training dataset (n = 2,804).

Model testing

RFC was compared with three widely used machine learning models, including DT, K-Nearest Neighbor (KNN), and Support Vector Machine (SVM), to prove its superiority. Generally, RFC performed better for most measures, while the recall of SVM was higher than that of RFC. More details are given in Table 4. The result of RFC prediction on the testing dataset is illustrated in a confusion matrix (Figure 3). In the testing dataset, 592 observations were classified as a positive label and 109 as negative. The true positive and false negative rates were 0.94 and 0.64, respectively. The recall of the RFC model was 0.94, and the F1-score was 0.84; the accuracy was similar to the precision (0.75). Also, the AUC of the RFC model was 0.65. The AP of the model was 0.84, which is similar to the average value of cross-validation. More details are shown in Figure 4.

TABLE 4

Table 4. Model performance on testing dataset.

FIGURE 3

Figure 3. Confusion matrix of RFC on testing dataset.

FIGURE 4

Figure 4. Discriminatory ability of Random Forest Classifier. (A) Receiver operating characteristic curve. (B) Two-class Precision-Recall curve.

Feature importance

The feature importance was computed as the (normalized) total reduction of the criterion brought by that feature, also known as the Gini importance (42). It was indicated that features from measure data were much more important than others. Attitude (0.219) was the most significant factor, followed by practice (0.1840), confidence (0.182), and knowledge (0.131). Feature importance of profession title, “Working in hospice care now,” location of the medical institution, and occupation 2 (Doctor) were 0.030 and over. All details are given in Figure 5.

FIGURE 5

Figure 5. Feature importance. It shows how important the value was in the model.

Discussion

Model performance

The result of RFC showed high sensitivity, suggesting that a majority of the samples were correctly classified into a positive label. However, the AUC (0.65) was still far from that of a perfect classification. It is considered that the class imbalance of the target variable caused the capacity of the discrimination to be poor, suggesting that we could not evaluate our classification models completely based on the accuracy and AUC. The Precision-Recall curve is a widely used measure of prediction for imbalanced classes (43). The AP was 0.84, which satisfied our expectations. Meanwhile, according to the study target of developing a productive HPC team, we paid more attention to whether the model contributed to identifying healthcare providers with positive behavioral intentions, focusing on the recall and the true positive rate of the model. Finally, the RFC was considered to have practical application value and could be a reasonable method for estimating individual behavioral intentions of HPC, organizing a productive team, and evaluating the ability to implement HPC.

Attitude was the most significant enabler

According to the outcome of RFC and BLR, attitude was the most significant enabler. A positive attitude was a crucial facilitator to improving the quality of HPC services and important for building teamwork and interpersonal relationship (44, 45). It was remarkable that healthcare providers who were divorced or widowed showed a negative attitude to work in HPC, owing to painful experiences that trigger negative emotions, nervousness, and other barriers. The correlative survey also highlighted that attitude toward caring for dying patients was mainly influenced by death anxiety of the healthcare provider (46). One would develop negative emotions, and death avoidance was the typical performance (47). Nurses, being the main providers of HPC, were the most probable participants experiencing death anxiety with a direct effect on their attitude toward caring for the dying (48). Further, death anxiety was also induced by high levels of burnout and occupational stress among healthcare providers. They may struggle to realize work value and career satisfaction (49). Therefore, healthcare providers must develop a positive attitude toward death to optimize the quality of HPC.

In our further analysis, aside from its own influence, attitude was associated with other features that affect behavioral intentions (50). Confidence was closely correlated with attitude. Healthcare providers with negative attitudes had insufficient confidence and rejected or escaped from caring for dying patients, which constituted a serious barrier to improving the quality of HPC (48). Also, it was noteworthy that healthcare providers still working in HPC showed a positive attitude. This shows that long-term practice or constant work in HPC could help service providers develop positive attitudes, which might release their death anxiety and discomfort for the death or dying patients (51).

Practice experience enhances the development of behavioral intentions

BLR and RFC provided similar evidence suggesting that practice experience would enhance the development of behavioral intentions. It has been illustrated that healthcare providers at CHCs show a lack of behavioral intentions, partly due to assigned work tasks by the government instead of by active participation, without sufficient financial support and an effective incentive mechanism. Worse still, in many CHCs, there are rare opportunities to practice since the public are unaware of HPC (52). Furthermore, confidence (self-efficacy) toward HPC would be influenced by the practice experience. Therefore, providing opportunities for practices could improve self-efficacy in providing HPC, suggesting the need for sustained government funding and long-term incentives, such as a reasonable payment system, to inspire the providers' enthusiasm now and in the long-term, and to strengthen public education of HPC. In addition, there should be a standardized training design, communication platform, or academic salon to encourage healthcare providers to discuss practice, personal involvement, and further research on HPC (53).

Knowledge is an undeniable facilitator

BLR and RFC highlighted that knowledge was an undeniable facilitator of behavioral intentions. The impact of knowledge includes two aspects: whether healthcare providers understand the concept of HPC and whether they have a good knowledge of the essential skills and apply them to HPC practice. For the former, we noticed that doctors showed negative behavioral intentions compared with other professionals. Most doctors would rarely consider HPC when cancer patients first visited; however, they agreed with the value of HPC (54). Doctors aim to cure patients or recover them from diseases to maintain life; however, some doctors still consider that HPC implies abandoning treatment. They tend to struggle to maintain life-prolonging treatment and gain poor satisfaction from HPC services (55). Therefore, it is urgent to disseminate the concept and benefits of HPC among healthcare providers and the public to help establish the concept of integrated life-cycle care and HPC for terminal patients.

For the latter, we found that healthcare providers working in the countryside showed negative behavioral intentions due to a lack of knowledge and essential skills of HPC. As previously reported, most providers lack the professional training, skills, and principles in HPC practice, particularly pain control, symptom management, and psychological care (56). Also, one-third of physicians had little knowledge of the HPC guidelines, and two-thirds did not know the Advance Care Planning manual (57). This might cause insufficient confidence, poor behavioral intentions, and impede practices and implementation of HPC.

Limitations

RFC outperformed other machine learning models and avoided the overfitting phenomenon; however, it is not a perfect classification. Therefore, we will keep working to develop more excellent methods. Additionally, evidence from Shanghai could not represent the situation of other provinces in China. Therefore, we will try to expand our samples to other areas to get stable evidence.

Conclusion

HPC has become a topic of interest owing to the rising burden of NCDs, population aging, and the COVID-19 pandemic. However, due to several barriers to implementing HPC, most patients cannot access the needed professional services. Therefore, it is significant to understand the real-world situation at the initial stage of the HPC movement in China, particularly the individual behavioral intentions to improve the accessibility of HPC. Our study was designed to predict the behavioral intentions of HPC providers using RF, which presented a first attempt in this area to automatically identify the relationship between behavioral intentions and other parameters of KAPHC. RFC showed high sensitivity, which suggested a well-trained RFC could help to estimate individual behavioral intentions to organize a productive team. Moreover, attitude is considered the most significant facilitator, implying the most efficient approach to encourage behavioral intentions. Reasonable HPC knowledge and practices in caring for terminal patients are of potential interest and stimulate the behavioral intentions of healthcare providers. Ultimately, the study has provided evidence that the government and HPC organizations should scale up standardized training, sustained financial support, and long-term incentives to attract providers to engage actively and enthusiastically in HPC. This will ensure its long-term sustainable development in China.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

This study was approved by the Ethics Committees of Shanghai Ninth People's Hospital (Ref: SH9H-2021-T11-1). Written informed consent from the participants was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

TC contributed to the original draft's conceptualization, formal analysis, and writing. YX contributed to the investigation and data curation. HZ contributed to visualization. XT contributed to the investigation and data curation. LJ contributed to writing—review and editing, supervision, and project administration. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by grants from the Nature Science Foundation of Shanghai (No. 22ZR1461400), Humanities and Social Science Research Planning Fund of the Ministry of Education (No. 20YJAZH045), Philosophy and Social Science Planning Project of Shanghai (No. 2019BGL032), and Pujiang Talent Program of Shanghai (No. 2019PJC099).

Acknowledgments

We gratefully acknowledge the participation of all healthcare institutions and providers. Additionally, we would like to thank Editage (www.editage.cn) for English language editing.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. United Nations Department Department of Economic Social Affairs Population Division. World Population Ageing 2019. New York: United Nations Press (2019). Available online at: https://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopulationAgeing2019-Report.pdf (accessed December 20, 2020).

2. Connor SR. Global Atlas of Palliative Care. 2nd ed. London: Worldwide Palliative Care Alliance Press (2020).

PubMed Abstract

3. World Health Organization. Why Palliative Care is an Essential Function of Primary Health Care. Technical Series on Primary Health Care. (2018). Available online at: https://www.who.int/docs/default-source/primary-health-care-conference/palliative.pdf?sfvrsn=ecab9b11_2&ua=1 (accessed June 20, 2021).

4. Bajwah S, Oluyase AO, Yi D, Gao W, Evans CJ, Grande G, et al. The effectiveness and cost-effectiveness of hospital-based specialist palliative care for adults with advanced illness and their caregivers. Cochr Datab Syst Rev. (2020) 2020:CD012780. doi: 10.1002/14651858.CD012780.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Higginson IJ, Yi D, Johnston BM, Ryan K, McQuillan R, Selman L, et al. Associations between informal care costs, care quality, carer rewards, burden and subsequent grief: the international, access, rights and empowerment mortality follow-back study of the last 3 months of life (IARE I study). BMC Med. (2020) 18:344. doi: 10.1186/s12916-020-01768-7

PubMed Abstract | CrossRef Full Text | Google Scholar

6. World Health Organization. WHO Health Topics. Palliative Care. (2021). Available online at: https://www.who.int/health-topics/palliative-care (accessed March 10, 2021).

7. World Health Organization. Newsroom. Fact Sheets. Detail. Palliative Care. (2021). Available online at: https://www.who.int/news-room/fact-sheets/detail/palliative-care (accessed March 10, 2021).

8. Zheng XY, Pan SH, Chen WQ, Chen CQ, Chen YX. Survey on residents' cognition status and choice intention to hospice care. Chin Med Ethics. (2021) 34:1003–9. doi: 10.12026/j.issn.1001-8565.2021.08.18

CrossRef Full Text

9. Pan QN, Huang HQ, Tao PY, Chen L, Li XH, Pan X, et al. Survey on knowledge, belief and practice of hospice care among hospice care workers: the influencing factors. J Nurs Sci. (2021) 36:20–22+26. doi: 10.3870/j.issn.1001-4152.2021.15.020

CrossRef Full Text

10. Zimmermann C, Swami N, Krzyzanowska M, Leighl N, Rydall A, Rodin G, et al. Perceptions of palliative care among patients with advanced cancer and their caregivers. Can Med Assoc J. (2016) 188:E217–E27. doi: 10.1503/cmaj.151171

PubMed Abstract | CrossRef Full Text | Google Scholar

11. World Health Organization. Integrating Palliative Care and Symptom Relief into Primary Health Care: A WHO Guide for Planners, Implementers and Managers. Switzerland: World Health Organization Press (2018). Available online at: https://apps.who.int/iris/bitstream/handle/10665/274559/9789241514477-eng.pdf?ua=1 (accessed June 20, 2021).

12. Teng XH, Tang MC, Jing LM, Xu YF, Shu ZQ. Healthcare provider knowledge, attitudes, and practices in hospice care and their influencing factors: a cross-sectional study in Shanghai. Int J Health Policy. (2022). doi: 10.34172/IJHPM.2022.6525. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Nedjat-Haiem FR, Cadet TJ, Amatya A, Mishra SI. Healthcare providers' attitudes, knowledge, and practice behaviors for educating patients about advance directives: a national survey. Am J Hosp Palliat Me. (2019) 36:387–95. doi: 10.1177/1049909118813720

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Chacko SA. Assessment of knowledge and attitude on palliative care among student nurses in selected college of nursing, New Delhi. Int J Nurs Midwif Res. (2021) 07:36–41. doi: 10.24321/2455.9318.202024

CrossRef Full Text | Google Scholar

15. Nair M, Kumar P, Mahajan R, Harshana A, Kurup KK, Moreto-Planas L, et al. Knowledge, attitudes, and practices regarding palliative care: a mixed-methods study from Bihar, India. J Palliat Care. (2021) 36:9–11. doi: 10.1177/0825859720946514

PubMed Abstract | CrossRef Full Text | Google Scholar

16. O'Shea ER, Mager D. End-of-life nursing education: enhancing nurse knowledge and attitudes. Appl Nurs Res. (2019) 50:151197. doi: 10.1016/j.apnr.2019.151197

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Pentaris P, Christodoulou P. Knowledge and attitudes of hospice and palliative care professionals toward diversity and religious literacy in cyprus: a cross-sectional study. J Palliat Med. (2021) 24:233–9. doi: 10.1089/jpm.2020.0011

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Tsao L, Slater SE, Doyle KP, Cuong DD, Khanh QT, Maurer R, et al. Palliative care-related knowledge, attitudes, and self-assessment among physicians in Vietnam. J Pain Symptom Manage. (2019) 58:1015–22 e10. doi: 10.1016/j.jpainsymman.2019.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Patel SJ, Chamberlain DB, Chamberlain JM. A machine learning approach to predicting need for hospitalization for pediatric asthma exacerbation at the time of emergency department triage. Acad Emerg Med. (2018) 25:1463–70. doi: 10.1111/acem.13655

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Scikit-learn: Machine Learning in Python. API Reference. sklearn.ensemble.RandomForestClassifier. (2021). Available online at: https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.RandomForestClassifier.html# (accessed July 20, 2021).

Google Scholar

21. Holodinsky JK, Yu AYX, Kapral MK, Austin PC. Using random forests to model 90-day hometime in people with stroke. BMC Med Res Methodol. (2021) 21:1–12. doi: 10.1186/s12874-021-01289-8

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Shu ZQ, Wang YT Li TT, Jing LM, Sun XM. Instrument development of healthcare providers' knowledge, attitude and practice of hospice care scale in China. Int J Health Plann Manage. (2021) 36:364–80. doi: 10.1002/HPM.3074

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ross MM, McDonald B, McGuinness J. The palliative care quiz for nursing (PCQN): the development of an instrument to measure nurses' knowledge of palliative care. J Adv Nurs. (1996) 23:126–37. doi: 10.1111/j.1365-2648.1996.tb03106.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Frommelt KH. Attitudes toward care of the terminally ill: an educational intervention. Am J Hosp Palliat Care. (2003) 20:13–22. doi: 10.1177/104990910302000108

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Liu WJ, Hu WY, Chiu YF, Chiu TY, Lue BH, Chen CY, et al. Factors that influence physicians in providing palliative care in rural communities in Taiwan. Support Care Cancer. (2005) 13:781–9. doi: 10.1007/s00520-005-0778-7

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Shimizu M, Nishimura M, Ishii Y, Kuramochi M, Kakuta N, Miyashita M. Development and validation of scales for attitudes, self-reported practices, difficulties and knowledge among home care nurses providing palliative care. Eur J Oncol Nurs. (2016) 22:8–22. doi: 10.1016/j.ejon.2016.02.009

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Xu YF, Jing LM, Li XY, Li SJ. Hospice and palliative care services development in Shanghai. Open Acc J Comp Alt Med. (2021) 3:165. doi: 10.32474/OAJCAM.2021.03.000165

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Caballero-Vazquez A, Romero-Bejar JL, Albendin-Garcia L, Suleiman-Martos N, Gomez-Urquiza JL, Canadas GR, et al. Risk factors for short-term lung cancer survival. J Clin Med. (2021) 10:519. doi: 10.3390/jcm10030519

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Scikit-Learn: Machine Learning in Python. Supervised learning. Decision Trees. (2021). Available online at: https://scikit-learn.org/dev/modules/tree.html#classification (accessed Jul 20, 2021).

Google Scholar

30. Elrod J, Mohr C, Wolff R, Boettcher M, Reinshagen K, Bartels P, et al. Using artificial intelligence to obtain more evidence? Prediction of length of hospitalization in pediatric burn patients. Front Pediatr. (2021) 8:613736. doi: 10.3389/fped.2020.613736

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Scikit-Learn: Machine Learning in Python. API Reference. sklearn.ensemble.RandomForestClassifier. (2021). Available online at: https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.RandomForestClassifier.html# (accessed July 20, 2021).

Google Scholar

32. Angraal S, Mortazavi BJ, Gupta A, Khera R, Ahmad T, Desai NR, et al. Machine learning prediction of mortality and hospitalization in heart failure with preserved ejection fraction. JACC Heart Fail. (2020) 8:12–21. doi: 10.1016/j.jchf.2019.06.013

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dai PP, Chang WF, Xin ZR, Cheng HW, Ouyang W, Luo AJ. Retrospective study on the influencing factors and prediction of hospitalization expenses for chronic renal failure in China based on random forest and LASSO regression. Front Public Health. (2021) 9:678276. doi: 10.3389/fpubh.2021.678276

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Goto T, Camargo CA, Faridi MK, Yun BJ, Hasegawa K. Machine learning approaches for predicting disposition of asthma and COPD exacerbations in the ED. Am J Emerg Med. (2018) 36:1650–4. doi: 10.1016/j.ajem.2018.06.062

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Scikit-Learn: Machine Learning in Python. API Reference. sklearn.preprocessing.OneHotEncoder. (2021). Available online at: https://scikit-learn.org/dev/modules/generated/sklearn.preprocessing.OneHotEncoder.html?highlight=one%20hot#sklearn.preprocessing.OneHotEncoder (accessed Jul 20, 2021).

Google Scholar

36. Scikit-Learn: Machine Learning in Python. Cross-Validation: Evaluating Estimator Performance. (2021). https://scikit-learn.org/dev/modules/cross_validation.html#cross-validation (accessed July 20, 2021).

Google Scholar

37. Feng Z, Shen H, Gao K, Su J, Yao S, Liu Q, et al. Machine learning based on clinical characteristics and chest CT quantitative measurements for prediction of adverse clinical outcomes in hospitalized patients with COVID-19. Eur Radiol. (2021) 31:7925–35. doi: 10.1007/s00330-021-07957-z

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Sills MR, Ozkaynak M, Jang H. Predicting hospitalization of pediatric asthma patients in emergency departments using machine learning. Int J Med Inform. (2021) 151:104468. doi: 10.1016/j.ijmedinf.2021.104468

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Mortazavi BJ, Downing NS, Bucholz EM, Dharmarajan K, Manhapra A, Li SX, et al. Analysis of machine learning techniques for heart failure readmissions. Circ-Cardiovasc Qual. (2016) 9:629. doi: 10.1161/Circoutcomes.116.003039

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Lorenzoni G, Sabato SS, Lanera C, Bottigliengo D, Minto C, Ocagli H, et al. Comparison of machine learning techniques for prediction of hospitalization in heart failure patients. J Clin Med. (2019) 8:jcm8091298. doi: 10.3390/jcm8091298

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Scikit-Learn: Machine Learning in Python. API Reference. sklearn.metrics.average_precision_score. (2021). Available online at: https://scikit-learn.org/dev/modules/generated/sklearn.metrics.average_precision_score.html#sklearn.metrics.average_precision_score (accessed July 20, 2021).

Google Scholar

42. Scikit-Learn: Machine Learning in Python. API Reference. sklearn.ensemble.RandomForestClassifier. property feature_importances_. (2021). Available online at: https://scikit-learn.org/dev/modules/generated/sklearn.ensemble.RandomForestClassifier.html?highlight=importance#sklearn.ensemble.RandomForestClassifier.feature_importances_ (accessed July 20, 2021).

Google Scholar

43. Saarela M, Ryynanen OP, Ayramo S. Predicting hospital associated disability from imbalanced data using supervised learning. Artif Intell Med. (2019) 95:88–95. doi: 10.1016/j.artmed.2018.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Cevik B, Kav S. Attitudes and experiences of nurses toward death and caring for dying patients in Turkey. Cancer Nurs. (2013) 36:E58–65. doi: 10.1097/NCC.0b013e318276924c

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Bartel AP, Freeman RB, Ichniowski C, Kleiner MM. Can a workplace have an attitude problem? Workplace effects on employee attitudes and organizational performance. Labour Econ. (2011) 18:411–23. doi: 10.1016/j.labeco.2011.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Peters L, Cant R, Payne S, O'connor M, McDermott F, Hood K, Morphet J. How death anxiety impacts nurses' caring for patients at the end of life: a review of literature. Open Nurs J. (2013) 7:14–21. doi: 10.2174/1874434601307010014

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Esnaashari F, Kargar FR. The relation between death attitude and distress: tolerance, aggression, and anger. Omega (Westport). (2018) 77:154–72. doi: 10.1177/0030222815593871

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Braun M, Gordon D, Uziely B. Associations between oncology nurses' attitudes toward death and caring for dying patients. Oncol Nurs Forum. (2010) 37:E43–9. doi: 10.1188/10.Onf.E43-e49

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Cheong CY, Ha NHL, Tan LLC, Low JA. Attitudes towards the dying and death anxiety in acute care nurses - can a workshop make any difference? A mixed-methods evaluation. Palliat Support Care. (2020) 18:164–9. doi: 10.1017/S1478951519000531

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Zhou Y, Li Q, Zhang W. Undergraduate nursing students' knowledge, attitudes and self-efficacy regarding palliative care in China: a descriptive correlational study. Nurs Open. (2021) 8:343–53. doi: 10.1002/nop2.635

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Barnett MD, Reed CM, Adams CM. Death attitudes, palliative care self-efficacy, and attitudes toward care of the dying among hospice nurses. J Clin Psychol Med Settings. (2021) 28:295–300. doi: 10.1007/s10880-020-09714-8

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Zhang RY, Miao J, Shen TH, Cao WQ. Status of the contents of the community palliative care services in Shanghai. Chin J General Pract. (2019) 17:1876–9. doi: 10.16766/j.cnki.issn.1674-4152.001079

CrossRef Full Text | Google Scholar

53. Mo LQ, Peng AN, Wang XY, Zhang ZH. A survey of the awareness of palliative care among community medical staff in Chengdu area. In: Proceedings of 2019 5th International Conference on Economics, Business, Finance, and Management (ICEBFM 2019). Francis Academic Press: London (2019). p. 563–70.

Google Scholar

54. Wu X, Zhang X, Zhang J, Cui X. Analysis of knowledge, attitude and behavior of oncology medical staff in palliative care. Ann Palliat Med. (2020) 9:985–92. doi: 10.21037/apm-20-851

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Deng HF, Zhang L, Yan WZ, Deng WY, He SL, Li SX, et al. Investigation and analysis of attitude toward hospice care in 282 oncology nurses. J Nurs. (2019) 26:49–52. doi: 10.16460/j.issn1008-9969.2019.04.049

CrossRef Full Text | Google Scholar

56. Achora S, Labrague LJ. An integrative review on knowledge and attitudes of nurses toward palliative care: implications for practice. J Hosp Palliat Nurs. (2019) 21:29–37. doi: 10.1097/NJH.0000000000000481

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Sorensen AR, Marsaa K, Prior TS, Bendstrup E. Attitude and barriers in palliative care and advance care planning in nonmalignant chronic lung disease: results from a Danish national survey. J Palliat Care. (2020) 35:232–5. doi: 10.1177/0825859720936012

PubMed Abstract | CrossRef Full Text | Google Scholar