The “Surprise Question” in Neurorehabilitation—Prognosis Estimation by Neurologist and Palliative Care Physician; a Longitudinal, Prospective, Observational Study

Background: The 12-months “surprise” question (12-SQ) for estimating prognosis and the need for integrating palliative care (PC) services has not yet been investigated for neurological patients.

Objective: Test the value of the 12-SQ on a sample of neurorehabilitation patients.

Methods: All patients newly registered in the Department of Neurorehabilitation, Dr. Becker Rhein-Sieg-Clinic (8/2016-03/2017) were asked to participate. The treating neurorehabilitation physicians (NP) and an external consulting PC physician (PCP) independently estimated patients' prognosis using the 12-SQ; while symptom burden was independently assessed using the standardized palliative outcome measurement HOPE-SP-CL, a set of additional neurological issues, and ECOG. Follow-up with consenting patients 12 months later was via telephone. Descriptive and inferential statistics were utilized in data analysis.

Results: Of 634 patients, 279 (44%) patients (male: 57.7%, female: 42.3%; mean age: 63 ± 14) (or, alternatively, their legal representative) consented and were assessed at baseline. Per patient NP and PCP both answered the 12-SQ with “Yes” (164), with “No” (42), or had different opinions (73). The “No” group displayed the highest symptom burden on all three measures for both disciplines. Overall, PCP scored higher (i.e., worse) than NP on all measures used. Follow-up was possible for 236 (drop-out: 15.4%) patients (deceased: 34 (14.4%), alive: 202 (85.6%)). Baseline scores on all measures were higher for deceased patients compared to those still living. Prognostic characteristics were: sensitivity: NP 50%, PCP 67.6%; specificity: NP 86.1%, PCP 70.3%, p < 0.001; positive predictive value: NP 37.8%, PCP 27.7%; negative predictive value: NP 91.1%, PCP 92.8%; area under the curve: NP 0.68, PCP 0.69; success rate: NP 80.9%, PCP 69.9%, p = 0.002. Regression analysis indicated that age, dysphagia and overburdening of family (NP answering the 12-SQ), dysphagia and rehabilitation phase (PCP answering the 12-SQ) were associated with increased likelihood of dying within 12 months. Without the 12-SQ as relevant predictor, age, dysphagia and ECOG were significant predictors (NP and PCP).

Conclusion: Combining the 12-SQ with a measurement assessing PC and neurological issues could potentially improve the 12-SQ's predictive performance of 12-month survival and help to identify when to initiate the PC approach. Clinical experiences influence assessment and prognosis estimation.

Introduction

For predicting the point at which to introduce palliative care for incurable cancer patients the German national S3 palliative guideline¹ recommends using the 12-months “surprise” question (12-SQ) (Would you be surprised if your patient would die within the next 12 months?). A “No” response (i.e., a poor prognosis) indicates that the assessor considers it a possibility that the patient could die within the next 12 months and, thus, palliative care should be initiated promptly.

The various disease entities to which the SQ has been applied thus far include cancer (1–5), chronic obstructive pulmonary disease (COPD) (6), nephrological diseases (7–13), as well as pediatric palliative care (14), intensive care (15), emergency care (16, 17), and in elderly care (17–19). The SQ has yet to be applied to neurological care where patients are characterized by different disease trajectories compared to other disease entities, especially cancer patients. Prognosis estimation is therefore challenging (20) and a suitable prognostic instrument would aid in estimating lifespan and indicating when best to initiate the palliative care approach for these patients.

A recent systematic review and meta-analysis revealed that the SQ it is not an ideal diagnostic tool for predicting one-year mortality, especially among non-cancer patients (21). Rather, the use of additional parameters seems warranted (22, 23).

Thus, in addition to the 12-SQ, supplementary assessment tools focusing on patients' symptom burden were employed for this study. Typical palliative care assessment tools, including the German Hospice and Palliative Care Evaluation initiative (HOPE = HOspiz- und PalliativErhebung) (24) and the internationally used palliative outcome scale (POS) (25, 26) were developed for patients suffering from later stage cancer diseases. This is not surprising, as presently, palliative and hospice care structures primarily care for advanced cancer patients² (27, 28), although the portion of cancer patients has slightly decreased (95% in 2005 vs. 76% in 2017 (see²) with respect to other disease entities such as neurological conditions, COPD, nephrological diseases or chronic heart failure (see²). For example, patients cared for in German palliative and hospice care structures suffering from nervous system diseases represented 4.8% in 2017 compared to only 0.8% in 2005 (see²). Despite this slight increase in the number of neurological patients in German palliative and hospice care, the current, rather small, percentage is still astonishing considering the great number and variety of neurological diseases, among them long-term neurological conditions (LTNC) which present with a high symptom burden. Given these cases are mostly incurable, symptom relief, and enhanced quality of life are the leading therapeutic goals in treating these patients, according to the World Health Organization (WHO)³. Using a combined neurorehabilitation and palliative care approach, neuropalliative rehabilitation for LTNC is on the way to becoming integrated into care in the UK (29–31), in contrast to Germany. Symptoms and complaints among neurological and cancer patients are in part fairly similar but may differ in their manifestation and certain issues clearly transcend those of cancer patients, presenting distinct challenges for such patients (32–40). Therefore, the neurorehabilitation study population was characterized utilizing a combination of a standard palliative care assessment tool (HOPE including ECOG) and an additional list of items representative for neurological disease entities as revealed from longstanding clinical experience, literature (32–40) and a previous study on glioblastoma (41). In addition to using the 12-SQ, this detailed characterization can help to identify further prognostic criteria of neurological patients, which may lead to improved prognostic accuracy of the 12-SQ, an approach in accordance with other studies commending additional tools other than the 12-SQ to predict mortality (22, 23).

The objective of this study was to investigate prognostic criteria for neurological patients. For the first time, the suitability of the 12-SQ for neurological patients was prospectively investigated, combining it with an assessment merging palliative care and neurological issues.

Patients were recruited from among those newly registered at a neurorehabilitation clinic providing care to a broad range of neurological disease entities. An important secondary goal was to examine whether the professional background—being a neurorehabilitation physician (NP) or being a palliative care physician (PCP) with no neurology background—played a role in assessment and prognosis estimation. This is a critical issue as a consultant palliative care service is not typically integrated in neurorehabilitation clinics and NP and their teams must make decisions on their own. On the other hand, the PCP might also care for neurological patients but only a small percentage of them are trained in neurology. Thus a complementary approach suggests itself, one that includes the professional assessment of both, NP and PCP.

In Summary, Aims of the Study Were

Primary Objective

Is the 12-SQ suitable for prognosis estimation with neurological patients?

Secondary Objectives

Does prognosis estimation depend on the physicians' background (NP vs. PCP)?

How is the study population characterized and assessed by both, NP and PCP?

How are the patients who died within this 12 month period actually characterized? Can factors be deduced which would help estimate the prognosis of these patients alone or in combination with the 12-SQ?

Materials and Methods

Study Design

This is a longitudinal, prospective, observational study. The recruitment period encompassed August 10, 2016–March 10, 2017. The follow-up period extended until March 10, 2018; 12 months later.

Study Participants

All newly admitted patients (permitted age range 18–100 years, all genders) then in treatment at the Dr. Becker Rhein-Sieg-Klinik, Department of Neurorehabilitation (phase B, C, D; a German classification system characterizing type and intensity of neurological rehabilitation) during the recruitment period were enrolled in the study after providing their informed written consent (or alternatively via their legal representative). The local ethics committees of the North Rhine Medical Chamber and of the University Hospital of Cologne approved the study (#16–118).

Data Collection

For quality assurance and to enable a patient-oriented care post-hospital discharge at the Dr. Becker Rhein-Sieg-Klinik, department for neurorehabilitation, an estimation of prognosis using the 12-SQ and an assessment of symptom burden was implemented into the clinical routine.

“Surprise”-Question

At time of admission, treating NP - neurologists with additional neurorehabilitation expertise, but no specialist training in palliative care—as well as an external consulting PCP—with no neurological training—responded independently to the 12-SQ. The PCP visited the Department of Neurorehabilitation once a week.

Assessment of Symptom Burden

Concurrently to answering the 12-SQ, both NP and PCP also independently assessed the symptom burden of the neurological patients utilizing the core documentation of the German Hospice and Palliative Care Evaluation initiative (HOPE), the HOPE symptom and problem checklist (HOPE-SP-CL) (24) including the Eastern Cooperative Oncology Group (ECOG) Performance Status scale. The HOPE-SP-CL consists of 17 items and assesses symptoms and problems representative for cancer patients in palliative care (24). Single items are scaled using a 4-point grading scale (0 = none, 1 = mild, 2 = moderate, 3 = severe) (possible total score: 0–51) (24). The ECOG Performance Status scale is a 5-point grading scale ranging from 0 (normal activity) to 4 (care-dependent, totally confined to bed).

A list of symptoms which might be of special importance for neurological patients who have or might develop palliative care needs was added to account for the particularities of the neurological patients' symptom burden (32–40). This “neuro supplement” was derived from clinical experience and existing literature (32–40). Augmenting this was a preliminary study (41) on assessing palliative care issues utilizing standardized outcome measurements (HOPE-SP-CL (24, 42), the POS (palliative outcome scale) (25, 26) as well as an open interview part which included symptoms not covered by these assessment tools. The neuro supplement scale derived from this comprises 13 items. Following the HOPE-SP-CL scale, single items of the neuro supplement are scaled using a 4-point grading Likert scale (0 = none, 1 = mild, 2 = moderate, 3 = severe) (possible total score: 0–39). All 3 scales (HOPE-SP-CL, ECOG, neuro supplement) combined in this study result in a possible total score ranging from 0 to 94.

Follow-Up

Twelve months after answering the 12-SQ, patients (or alternatively their legal representative) were contacted via telephone by NP (ME) or PCP (AK) to find out whether patients were still alive.

Statistical Analysis

Distribution of age, gender, rehabilitation phase, main, and secondary diagnoses, results from the clinical assessment (12-SQ-answer, HOPE-SP-CL, ECOG, neuro supplement) were analyzed descriptively to characterize the study population at baseline.

The classification of patients was done in two separate steps. First, we used each physician's independent response to the 12-SQ, assigning patients to either the “Yes” group or the “No” group. Next, we combined the physicians' responses and allocated patients into three individual groups. Those who were given a good prognosis by both physicians were classified into the 12-SQ “Yes” group, those given a poor prognosis by both physicians were categorized as the 12-SQ “No” group, and those with contrary ratings were classified as the 12-SQ “Discordant”-group. This classification into three groups allowed us to characterize the study population (statistical details below) when the consensus of both physicians was used (secondary objective).

Prior to all analyses, the Kolmogorov-Smirnov Test was applied to assess normality for all relevant variables.

To investigate whether the 12-SQ can be used as a prognostic indicator for neurological patients (primary objective), the predictive power of the 12-SQ was determined using receiver operating characteristics (ROC) curves to assess sensitivity (proportion of patients who died within 12 months and were given a poor prognosis), specificity (proportion of patients who survived over 12 months and were given a good prognosis), positive predictive value (PPV, proportion of poor prognoses correctly predicting death within 12 months), negative predictive value (NPV, proportion of good prognoses correctly predicting survival over 12 months), and the area under the curve (AUC, function of both sensitivity and specificity measuring the predictive accuracy). Similarly, we examined the success rates of both physicians; defined as percentage of correct predictions accounting for all possible outcomes. The differences in prognostic accuracy proportions between the NP and PCP were assessed with the McNemar χ² test as appropriate (secondary objective) (43).

Group differences of demographic and clinical data were tested with the Mann-Whitney-U test and the Kruskall-Wallis test (for two and three groups, respectively) for continuous measures and a χ² test for dichotomous measures. Post-hoc tests were corrected for multiple comparisons using the false discovery rate (FDR) at p < 0.05 (44).

To determine independent predictors of 12-month mortality (secondary objective), binary logistic regression analyses were performed for both NP and PCP. In addition to the 12-SQ, age, gender, main and secondary diagnoses, frequency of secondary diagnoses, rehabilitation phase, HOPE-SP-CL single items, neuro supplement single items and the ECOG score were included in the model. A univariate regression was constructed first. Resulting predictors with a p-value of < 0.1 were then included in the multivariable regression. To determine which variables best predicted 12-month mortality in the presence and absence of the 12-SQ, we selected statistically significant predictors of the multivariable regression (at p < 0.05) for the final model and compared their prognostic accuracy indices to those of the 12-SQ as a stand-alone predictor.

Statistical analysis was performed using the Statistical Package for the Social Sciences (SPSS) software (v. 25, Inc, Chicago, IL).

Results

Study Participation and Follow-Up

From August 10, 2016 through March 10, 2017, 634 patients were admitted to the Dr. Becker-Rhein-Sieg-Clinic Nümbrecht, Department of neurorehabilitation (Figure 1). Of this total, 137 (21.6% of 634) could not be included into the current study due to the restricted personnel resources and tightly packed clinical routine processes. Another 218 (34.4% of 634) patients had incomplete data, i.e., either the patients (or alternatively their legal representative) did not give their informed written consent or the physicians' assessment was not attainable because of the demanding clinical routines of the physicians or patients. The remaining 279 patients were independently assessed by both NP and PCP who concordantly estimated 164 (58.8%) patients with a good prognosis (both 12-SQ “Yes”) and 42 (15.1%) patients with a poor prognosis (both 12-SQ “No”). 73 (26.2%) patients were estimated discordantly (one, either NP or PCP, answered 12-SQ “Yes”, while the other responded “No”). A total of 43 out of 279 (equaling a drop-out rate of 15.4%) patients (or alternatively their legal representative) could not be followed-up due to unattainability via the phone (Figure 1). Complete data sets were obtained from 236 patients (37.2% of 634).

FIGURE 1

Figure 1. Flow-chart of study participation and follow-up.

Characteristics of Study Participants at Baseline

Demographic information of the included 279 patients can be found in Table 1 (male: 57.7%, female: 42.3%, male/female ratio: 1.4; mean age: 63 ± 14). Main diagnoses were grouped into 15 categories, secondary diagnoses were divided into seven groups (Table 1). Distribution of secondary diagnoses were as follows: 19.6% had no secondary diagnosis, 33.8% had secondary diagnoses in one category, 25.5% in two categories, 14.7% in three categories, 5.8% in four categories, and 0.7% in five categories, respectively (Table 1).

TABLE 1

Table 1. Characteristics of study participants at baseline.

Characteristics of study participants at baseline as assessed by NP and PCP, respectively, utilizing HOPE-SP-CL, neuro supplement, and ECOG are presented in Table 2. Patients were given a higher score on all three measures when assessed by the PCP (all p-values < 0.025), except for feeling depressed and anxiety in the “No” group.

TABLE 2

Table 2. Assessment of patients' symptom burden (HOPE-SP-CL, ECOG, neurological symptoms) by neurorehabilitation physician (NP) and palliative care physician (PCP), respectively, for the patients who were concordantly estimated to have a good prognosis (NP and PCP, both answered 12-SQ with “Yes”), for the patients who were concordantly assessed as having a poor prognosis (neurologist and PC physician, both answered 12-SQ with “No”) and for the patients whose prognosis was discordantly estimated by neurologist and palliative care physician (NP answered 12-SQ with “No” and PCP with “Yes” and vice versa, respectively).

Significant group differences (12-SQ “Yes” by both NP and PCP; 12-SQ “No” by both NP and PCP; 12-SQ “Yes”/“No” NP and PCP discordant) were found for all but the following characteristics: pain, feeling depressed, anxiety, tension, symptoms of intracranial pressure, epileptic seizures, spasticity (all p-values < 0.031). As expected, post-hoc analyses showed that patients in the “No” group were evaluated with a higher symptom burden than patients in both the “Yes” and the “Discordant” group, and patients in the “Discordant” group were evaluated as worse than patients in the “Yes” group.

Characteristics of Deceased and Surviving Patients as Assessed at Baseline

Of the 115 patients assessed “No” on the 12-SQ by at least one discipline (Figure 1) 26 had died within the year. At the 12 months follow-up a total of 34 patients had died (also encompassing eight patients estimated as “Yes” at baseline on the 12-SQ).

Table 3 summarizes the characteristics and differences of patients still alive at the time of 12-month follow-up (N = 202, 85.6%) and those deceased after 12 months (N = 34, 14.4%). The deceased were significantly older, more often in Rehabilitation phase B, less often in Rehabilitation phase D, and suffered significantly more often from malignancies (except for primary brain tumors) (all p-values < 0.001). With regards to our clinical outcome measures, deceased patients were evaluated with a higher symptom burden compared to patients still alive after 12 months (all p-values < 0.030) (Table 3).

TABLE 3

Table 3. Characteristics of patients deceased after 12 months (N = 34) and those still alive (N = 202).

Prognosis Estimation

Comparison of prognosis estimation via 12-SQ as diagnostic tool revealed an increased number of good prognoses (12-SQ “Yes”) compared to poor prognoses (12-SQ “No”) for both PCP (p < 0.001) and NP (p < 0.001). The PCP estimated more patients with a poor prognosis (12-SQ “No”) (N = 95) than did the NP (N = 62) (p = 0.008). Also, he offered a worse clinical assessment of patients compared to the NP. This difference is statistically significant for the total sample, the concordant “Yes” group and the discordant “Yes (NP)/No (PCP)” group (each p < 0.001) (Table 4).

TABLE 4

Table 4. Prognosis estimation of NP and PCP.

Prognostic accuracy indices for both disciplines are summarized in Table 5, the corresponding frequency distribution can be found in Tables 6–7. Sensitivity of the 12-SQ as stand-alone predictor was poor. While we observed a higher sensitivity for responses of the PCP relative to treating NP, this difference did not achieve statistical significance. In contrast, specificity of the 12-SQ was significantly higher when estimated by NP compared to PCP [${\chi }_{(1,\mathit{\text{N}}=194)}^2=14.58$, p < 0.001]. There were no statistically significant differences between physicians for PPV, NPV, or AUC.

TABLE 5

Table 5. Prognostic accuracy indices, 95% confidence intervals are displayed in brackets.

TABLE 6

Table 6. Frequency table for prognosis estimation by the neurorehabilitation physician.

TABLE 7

Table 7. Frequency table for prognosis estimation by the palliative care physician.

The combined “yes” and “no” success rate was high, with a significant difference between NP and PCP [${\chi }_{(1,\mathit{\text{N}}=236)}^2$ = 9.47, p = 0.002] (Table 5). The success rate for the “Yes” group was also significantly higher for NP relative to PCP [${\chi }_{(1,\mathit{\text{N}}=236)}^2$ = 17.80, p < 0.001]. Conversely, the success rate for giving a poor prognosis did not differ between physicians.

Regression analysis showed that age (p = 0.015), dysphagia (p = 0.006), and overburdening of the family (p = 0.036) were associated with an increased likelihood of dying at 12 months when the NP responded to the 12-SQ (Figure 2, Table 8). Overall classification was 80.3% accurate.

FIGURE 2

Figure 2. ROC curves showing the trade-off between sensitivity and specificity for all significant predictors after multivariable regression with (Model 1) and without (Model 2) the NP's response to the 12-SQ. Classification accuracy is higher for both models compared to the 12-SQ as stand-alone predictor (Model 0), though these differences did not reach statistical significance.

TABLE 8

Table 8. Binary logistic regression to predict 12-month mortality as assessed by the neurorehabilitation physician.

When patients were assessed by the PCP, the overall predictive accuracy of the model was 79.9%. Response to the 12-SQ (p = 0.014), dysphagia (p = 0.041), and rehabilitation phase (p = 0.014) were statistically associated with 12-month mortality. Patients in the “No” group were 3 times more likely to die than patients in the “Yes” group. Rehabilitation phase also predicted the likelihood of dying at 12 months with patients in phase B registering as 7.3 times more likely to die than patients in phase D (p = 0.005), and patients in phase C being 2.8 times more likely to die compared to patients in phase D (p = 0.041) (Figure 3, Table 9).

FIGURE 3

Figure 3. ROC curves showing the trade-off between sensitivity and specificity for all significant predictors after multivariable regression with (Model 1) and without (Model 2) the PCP's response to the 12-SQ. Classification accuracy is higher for both models compared to the 12-SQ as stand-alone predictor (Model 0), though these differences did not achieve statistical significance.

TABLE 9

Table 9. Binary logistic regression to predict 12-month mortality as assessed by the Palliative Care Physician.

Without the 12-SQ as relevant predictor, age (NP: p = 0.038; PCP: p = 0.026) and dysphagia (NP: p = 0.012, PCP: p = 0.029) remained significant predictors, irrespective of the physicians' medical background. In addition, for both the NP and the PCP, an increased ECOG score was significantly related to an increased risk of dying (NP: p = 0.003; PCP: p = 0.005) (Figures 2–3, Tables 8, 9). When assessed by the PCP, the model showed 86.9% overall classification accuracy, which increased to 89% when assessed by the NP.

Discussion

According to literature search this is the first study investigating prognosis estimation using 12-SQ and assessment of palliative care symptoms supplemented by neurological items, as rated by NP and PCP, respectively, in a sample of neurorehabilitation patients.

Prognosis estimation in this patient group proved challenging when utilizing 12-SQ as a single tool, which was reflected in poor prognostic accuracy indices, found also for other non-cancer diseases (21, 23). However, in our study, answering 12-SQ “No” pointed to physicians' expectation of poor prognosis as both treating NP as well as the PCP evaluated the 12-SQ “No” group consistently with the highest symptom burden. Overall, treating NP assessed patients better (meaning lower scores on the utilized measures) than the PCP. A potential explanation might be the clinical background of assessors with PCP primarily caring for the potential of general deterioration and the end of life and the NP being more concerned with recovery and restitution. Seemingly combined expertise might be needed for a balanced and accurate estimation.

In our study, accurate prediction for patients at increased risk of dying was especially low for NP. Accordingly, the NP demonstrated higher accuracy for predicting whether patients would still be alive after 12 months compared to PCP. Our results suggest the use of “12-SQ2”: “Would I be surprised if this patient is still alive after twelve months?” (45) for physicians with a background in neurology or a combination of the original 12-SQ and the 12-SQ2, which has been piloted in a sample of general practitioners (45, 46).

Significantly, the 12-SQ was not originally developed for an accurate prognosis in the prediction of death, but to identify patients in need of palliative care (1–19). In specialties such as neurorehabilitation the implementation of the 12-SQ in combination with a palliative care assessment into the clinical routine—as in our study—might help sensitize healthcare professionals toward palliative care issues like initiating conversation on advanced care planning or prognosis or integrating additional services like palliative and hospice care services if needed. Currently, this approach is not yet well recognized in German neurorehabilitation and integrative prognostic studies may serve to help change this, an eventual consequence which would be beneficial to both patients and the caregivers involved in neurorehabilitation. The need for such a multi-disciplinary neuropalliative rehabilitation approach has already been highlighted and recommended in the UK's National Service Framework for Long-term (Neurological) Conditions (29–31) but has not been consistently pursued in neurorehabilitation in Germany so far.

In a recent study, the 12-SQ was combined with further clinical parameters to better identify patients with palliative care needs and aid in prognosis estimation (22). Our study corroborates the importance of bringing in additional clinical assessments to the 12-SQ, i.e., HOPE-SP-CL, neuro supplement, ECOG, diagnoses, age, gender, rehabilitation phase, to establish a broader basis for estimation of prognosis and palliative care needs.

With the help of the additional data we were able to identify several items (HOPE-SP-CL as well as neuro supplement as well as ECOG) which were scored significantly higher at baseline (meaning worse) for the group of patients who died after 12 months compared to those still alive. This speaks in favor of these measurements being suitable to assess patients' deteriorating general health condition. Moreover, the regression identified three factors (age, ECOG, dysphagia) which might help to predict one-year mortality in our sample of neurorehabilitation patients. These three factors are all reasonable indicators for a worsened overall condition. As anticipated, increased age is a risk factor for dying, even more so when seriously ill. Second, an increasing ECOG score in patients indicates decreasing, i.e., worse, functionality in all daily activities. Lastly, dysphagia has been identified as a critical prognostic factor in neurological patients, especially those suffering from stroke and neurodegenerative disorders (20, 47, 48). In the rehabilitation setting mortality risk increased by a factor of 13 for patients suffering from dysphagia (47). Depending on subtypes, patients suffering from progressive supranuclear palsy or multiple system atrophy died 2–24 months after developing severe dysphagia (48). Potential reasons for dysphagia being associated with a poor prognosis might be the development of serious complications like aspiration pneumonia (20, 47, 48).

As the 12-SQ is a commonly used tool for estimation of prognosis—even if poor when used as the only instrument—and for initiating palliative care in cancer and non-cancer patients (1–19), we investigated whether adding further clinical characteristics to the 12-SQ would improve the overall predictive power. Again, our results indicate that age and dysphagia, as well as rehabilitation phase and overburdening of the family in combination with the 12-SQ have great prognostic value in estimating prognosis and thus identifying patients in need of palliative care. These two additional factors can be interpreted similarly to the ECOG: Patients in rehabilitation phases C and B suffer from a more serious illness with decreased functionality compared to phase D and “overburdening of family” also indicates patients' health deterioration. It is well known that as patients' health condition worsens, family caregivers physically and psychologically reach their limits (49–52).

Limitations

Of our initial sample of 634 patients only 236 (37%) could be included and later followed up. This proportion is quite good for a palliative care study, but generalizability remains limited as we were unable to present a full data set. Moreover, study participants attending rehabilitation phase B (i.e., seriously ill patients) were represented to a lesser degree than patients in rehabilitation phase C or D. One potential reason might be the increased difficulty in obtaining consent (seriously ill, legal representative, etc.). Of the 236 included and followed-up patients 14% died within one year. Despite similar incidences of death reported in other studies investigating the 12-SQ (21) this is a moderate to small fraction complicating the interpretation of prognostic accuracy indices. The neurorehabilitation population investigated was quite heterogeneous. Group sizes of the different main diagnoses groups were unequal ranging from 47% (ischemic stroke, largest group) to 0.2% (epilepsy, dementia syndrome, and hypoxic brain injury, respectively) and thus, a sound subgroup analysis was not possible. At least from results of this study, we cannot conclude whether the 12-SQ and identified risk factors may be of differing predictive accuracy with respect to special disease entities s (e.g., ALS representing a progressive disorder vs. ischemic stroke normally representing a monophasic illness). One caveat to the interpretation of our results is that various NPs (each time the respective treating NP) evaluated the patients while there was a single, external, consulting PCP assessing the patients, so that systematic assessment bias for the PCP could not be averaged out. That clinical background influenced 12-SQ estimation was also apparent amongst the rather large group of patients who were discordantly judged using the 12-SQ (26.2%) thereby reducing the number of unambiguously assigned patients.

Conclusion

Prognosis estimation of neurological patients is challenging and thus, identifying the right point in time to integrate the palliative care approach for neurological patients remains difficult. Implementing an assessment tool into the care of these patients - in the current study with a sample of neurorehabilitation patients - combining the 12-SQ with palliative care and neurological items might improve predictive performance of 12 months survival and thus identify an appropriate, sufficient time to initiate the palliative care approach and services if needed. Factors improving predictive accuracy (with and without the 12-SQ) were rehabilitation phase, dysphagia, age, overburdening of family and ECOG. Professional background influences assessment and prognosis estimation.

Author Contributions

ME, HG, RV, AK, and IB planned the study. ME and AK recruited and assessed patients. KD, AK, IB, and HG analyzed the data. HG, KD, AK, ME, IB, and RV wrote and corrected the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The handling editor is currently organizing a Research Topic with one of the authors RV, and confirms the absence of any other collaboration.

Acknowledgments

We thank all study participants and clinical personnel of the Dr. Becker Rhein-Sieg-Clinic, especially Britta Michler and Marion Birkholz for their valued assistance.

Footnotes

1. ^http://www.awmf.org/uploads/tx_szleitlinien/128-001OLl_S3_Palliativmedizin_2015-07.pdf (accessed 29.05.2018)

2. ^https://www.hope-clara.de/download_1/ (accessed 29.05.2018)

3. ^http://www.who.int/cancer/palliative/definition/en/ (accessed 29.05.2018)

Abbreviations

12-SQ, twelve-months “surprise”question; PC, palliative care; NP, neurorehabilitation physicians; PCP, palliative care physician; HOPE = HOspiz- und PalliativErhebung, German Hospice and Palliative Care Evaluation initiative; HOPE-SP-CL, HOPE symptom and problem checklist; ECOG, Eastern Cooperative Oncology Group; POS, palliative outcome scale; COPD, chronic obstructive pulmonary disease; LTNC, long-term neurological conditions; WHO, World Health Organization; ROC, receiver operating characteristics; PPV, positive predictive value; NPV, negative predictive value; AUC, area under the curve; FDR, false discovery rate; SPSS, Statistical Package for the Social Sciences; SD, standard deviation; ADLs, activity of daily living; OR, odds ratio.

References

1. Moss AH, Lunney JR, Culp S, Auber M, Kurian S, Rogers J, et al. Prognostic significance of the “surprise” question in cancer patients. J Palliat Med. (2010) 13:837–40. doi: 10.1089/jpm.2010.0018

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Moroni M, Zocchi D, Bolognesi D, Abernethy A, Rondelli R, Savorani G, et al. The ‘surprise’ question in advanced cancer patients: a prospective study among general practitioners. Palliat Med. (2014) 28:959–64. doi: 10.1177/0269216314526273

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Rhee J, Clayton JM. The ‘surprise’ question may improve the accuracy of GPs in identifying death in patients with advanced stage IV solid-cell cancer. Evid Based Med. (2015) 20:71. doi: 10.1136/ebmed-2014-110114

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hui D. Prognostication of survival in patients with advanced cancer: predicting the unpredictable? Cancer Control (2015) 22:489–97. doi: 10.1177/107327481502200415

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Hamano J, Morita T, Inoue S, Ikenaga M, Matsumoto Y, Sekine R, et al. Surprise questions for survival prediction in patients with advanced cancer: a multicenter prospective cohort study. Oncologist (2015) 20:839–44. doi: 10.1634/theoncologist.2015-0015

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Duenk RG, Verhagen SC, Janssen MA, Dekhuijzen RP, Vissers KC, Engels Y, et al. Consistency of medical record reporting of a set of indicators for proactive palliative care in patients with chronic obstructive pulmonary disease. Chron Respir Dis. (2017) 14:63–71. doi: 10.1177/1479972316661922

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Moss AH, Ganjoo J, Sharma S, Gansor J, Senft S, Weaner B, et al. Utility of the “surprise” question to identify dialysis patients with high mortality. Clin J Am Soc Nephrol. (2008) 3:1379–84. doi: 10.2215/CJN.00940208

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Pang WF, Kwan BC, Chow KM, Leung CB, Li PK, Szeto CC. Predicting 12-month mortality for peritoneal dialysis patients using the “surprise” question. Perit Dial Int. (2013) 33:60–6. doi: 10.3747/pdi.2011.00204

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Da Silva Gane M, Braun A, Stott D, Wellsted D, Farrington K. How robust is the ‘surprise question’ in predicting short-term mortality risk in haemodialysis patients? Nephron Clin Pract. (2013) 123:185–93. doi: 10.1159/000353735

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Amro OW, Ramasamy M, Strom JA, Weiner DE, Jaber BL. Nephrologist-facilitated advance care planning for hemodialysis patients: a quality improvement project. Am J Kidney Dis. (2016) 68:103–9. doi: 10.1053/j.ajkd.2015.11.024

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Javier AD, Figueroa R, Siew ED, Salat H, Morse J, Stewart TG, et al. Reliability and utility of the surprise question in CKD stages 4 to 5. Am J Kidney Dis. (2017) 70:93–101. doi: 10.1053/j.ajkd.2016.11.025

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Malhotra R, Tao X, Wang Y, Chen Y, Apruzzese RH, Balter P, et al. Performance of the surprise question compared to prediction models in hemodialysis patients: a prospective study. Am J Nephrol. (2017) 46:390–6. doi: 10.1159/000481920

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Salat H, Javier A, Siew ED, Figueroa R, Lipworth L, Kabagambe E, et al. Nephrology provider prognostic perceptions and care delivered to older adults with advanced kidney disease. Clin J Am Soc Nephrol. (2017) 12:1762–70. doi: 10.2215/CJN.03830417

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Burke K, Coombes LH, Menezes A, Anderson AK. The ‘surprise’ question in paediatric palliative care: a prospective cohort study. Palliat Med. (2018) 32:535–42. doi: 10.1177/0269216317716061

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hadique S, Culp S, Sangani RG, Chapman KD, Khan S, Parker JE, et al. Derivation and validation of a prognostic model to predict 6-month mortality in an intensive care unit population. Ann Am Thorac Soc. (2017) 14:1556–61. doi: 10.1513/AnnalsATS.201702-159OC

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Haydar SA, Almeder L, Michalakes L, Han PKJ, Strout TD. Using the surprise question to identify those with unmet palliative care needs in emergency and inpatient settings: what do clinicians think? J Palliat Med. (2017) 20:729–35. doi: 10.1089/jpm.2016.0403

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ouchi K, Jambaulikar G, George NR, Xu W, Obermeyer Z, Aaronson EL, et al. The “surprise question” asked of emergency physicians may predict 12-month mortality among older emergency department patients. J Palliat Med. (2018) 21:236–40. doi: 10.1089/jpm.2017.0192

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Lilley EJ, Gemunden SA, Kristo G, Changoor N, Scott JW, Rickerson E, et al. Utility of the “surprise” question in predicting survival among older patients with acute surgical conditions. J Palliat Med. (2017) 20:420–3. doi: 10.1089/jpm.2016.0313

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Rice J, Hunter L, Hsu AT, Donskov M, Luciani T, Toal-Sullivan D, et al. Using the “surprise question” in nursing homes: a prospective mixed-methods study. J Palliat Care. (2018) 33:9–18. doi: 10.1177/0825859717745728

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Oliver D (ed.). End of Life Care in Neurological Disease. London: Springer (2013). doi: 10.1007/978-0-85729-682-5

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Downar J, Goldman R, Pinto R, Englesakis M, Adhikari NK. The “surprise question” for predicting death in seriously ill patients: a systematic review and meta-analysis. CMAJ (2017) 189:E484–93. doi: 10.1503/cmaj.160775

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Gómez-Batiste X, Martínez-Mu-oz M, Blay C, Amblàs J, Vila L, Costa X, et al. Utility of the NECPAL CCOMS-ICO(©) tool and the surprise question as screening tools for early palliative care and to predict mortality in patients with advanced chronic conditions: a cohort study. Palliat Med. (2017) 31:754–63. doi: 10.1177/0269216316676647

PubMed Abstract | CrossRef Full Text | Google Scholar

23. White N, Kupeli N, Vickerstaff V, Stone P. How accurate is the ‘surprise question’ at identifying patients at the end of life? A systematic review and meta-analysis. BMC Med. (2017) 15:139. doi: 10.1186/s12916-017-0907-4

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Stiel S, Pollok A, Elsner F, Lindena G, Ostgathe C, Nauck F, et al. Validation of the symptom and problem checklist of the German hospice and palliative care evaluation (HOPE). J Pain Symptom Manag. (2012) 43:593–605. doi: 10.1016/j.jpainsymman.2011.04.021

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Hearn J, Higginson IJ. Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative care core audit project advisory group. Qual Health Care (1999) 8:219–27. doi: 10.1136/qshc.8.4.219

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Bausewein C, Fegg M, Radbruch L, Nauck F, Mackensen S, Borasio GD, et al. Validation and clinical application of the German version of the palliative care outcome scale. J Pain Symptom Manag. (2005) 30:51–63. doi: 10.1016/j.jpainsymman.2005.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Golla H, Ebke M, Rolke R, Lorenzl S, Nacimiento W, Thiekötter T, et al. Integration of palliative and hospice care structures in the care of neurological patients: opinion of chief consultant neurologists in Germany. Fortschr Neurol Psychiatry (2016) 84:733–8. doi: 10.1055/s-0042-119027

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Golla H, Fink GR, Rolke R, Lorenzl S, Ebke M, Montag T, et al. New structures in neurology: palliative care for neurological patients. Neurol Int Open (2017) 1:E117–26. doi: 10.1055/s-0043-108438

CrossRef Full Text | Google Scholar

29. Turner-Stokes L, Sykes N, Silber E. Guideline Development Group. Long-term neurological conditions: management at the interface between neurology, rehabilitation and palliative care. Clin Med. (2008) 8:186–91. doi: 10.7861/clinmedicine.8-2-186

CrossRef Full Text | Google Scholar

30. Turner-Stokes L, Sykes N, Silber E, Khatri A, Sutton L, Young E. From diagnosis to death: exploring the interface between neurology, rehabilitation and palliative care in managing people with long-term neurological conditions. Clin Med. (2007) 7:129–36. doi: 10.7861/clinmedicine.7-2-129

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Khan F, Amatya B, Turner-Stokes L. Symptomatic therapy and rehabilitation in primary progressive multiple sclerosis. Neurol Res Int. (2011) 2011:740505. doi: 10.1155/2011/740505

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Higginson IJ, Hart S, Silber E, Burman R, Edmonds P. Symptom prevalence and severity in people severely affected by multiple sclerosis. J Palliat Care. (2006) 22:158–65.

PubMed Abstract | Google Scholar

33. Oliver DJ, Campbell C, O'Brien Sloan R, Sykes N, Tallon C, et al. Medication in the last days of life for motor neuron disease / amyotrophic lateral sclerosis. Amyotroph Lateral Scler. (2010) 11:562–4. doi: 10.3109/17482968.2010.493203

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Hensler M, Paul S, Abright C, Lorenzl S. Progressive supranuclear palsy: living environment of the patients in Germany. Nervenarzt (2011) 82:207–14. doi: 10.1007/s00115-010-3076-7

PubMed Abstract | CrossRef Full Text

35. Ostgathe C, Alt-Epping B, Golla H, Gaertner J, Lindena G, Radbruch L, et al. Non-cancer patients in specialized palliative care in Germany: what are the problems? Palliat Med. (2011) 25:148–52. 10.1177/0269216310385370. doi: 10.1177/0269216310385370

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Golla H, Galushko M, Pfaff H, Voltz R. Unmet needs of severely affected multiple sclerosis patients: the health professionals' view. Palliat Med. (2012) 26:139–51. doi: 10.1177/0269216311401465

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Lorenzl S, Nubling G, Perrar KM, Voltz R. Palliative treatment of chronic neurologic disorders. Handb Clin Neurol. (2013) 118:133–9. doi: 10.1016/B978-0-444-53501-6.00010-X

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Grehl T. Diagnostics and therapy for amyotrophic lateral sclerosis. Fortschr Neurol Psychiatry (2013) 81:592–603. doi: 10.1055/s-0033-1350151

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lorenzl S. Multimorbidity of neurological patients in palliative care units. Nervenarzt (2014) 85:409–16. doi: 10.1007/s00115-013-3938-x

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Basedow-Rajwich B, Koehler J. Besonderheiten der neuropalliativmedizinischen versorgung schwerst betroffener MS-Patienten. Akt Neurol. (2015) 42:11–5. doi: 10.1055/s-0034-1398524

CrossRef Full Text | Google Scholar

41. Golla H, Ale Ahmad M, Galushko M, Hampl J, Maarouf M, Schroeter M, et al. Glioblastoma multiforme from diagnosis to death: a prospective, hospital-based, cohort, pilot feasibility study of patient reported symptoms and needs. Support Care Cancer (2014) 22:3341–52. doi: 10.1007/s00520-014-2384-z

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Radbruch L, Nauck F, Ostgathe C, Elsner F, Bausewein C, Fuchs M, et al. What are the problems in palliative care? Results from a representative survey. Support Care Cancer (2003) 11:442–51. doi: 10.1007/s00520-003-0472-6

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Trajman A, Luiz RR. McNemar chi2 test revisited: comparing sensitivity and specificity of diagnostic examinations. Scand J Clin Lab Invest. (2008) 68:77–80. doi: 10.1080/00365510701666031

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Benjamini Y, Hochberg Y. Controlling the false discovery rate - a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol. (1995) 57:289–300.

Google Scholar

45. Weijers F, Veldhoven C, Verhagen C, Vissers K, Engels Y. Adding a second surprise question triggers general practitioners to increase the thoroughness of palliative care planning: results of a pilot RCT with cage vignettes. BMC Palliat Care (2018) 17:64. doi: 10.1186/s12904-018-0312-6

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Klok L, Engels Y, Veldhoven C, Rotar Pavlič D. Early identification of patients in need of palliative care in Slovenian general practice. Zdr Varst. (2018) 57:55–64. doi: 10.2478/sjph-2018-0008

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Cichero JA, Altman KW. Definition, prevalence and burden of oropharyngeal dysphagia: a serious problem among older adults worldwide and the impact on prognosis and hospital resources. Nestle Nutr Inst Workshop Ser. (2012) 72:1–11. doi: 10.1159/000339974

PubMed Abstract | CrossRef Full Text | Google Scholar

48. O'Sullivan SS, Massey LA, Williams DR, Silveira-Moriyama L, Kempster PA, Holton JL, et al. Clinical outcomes of progressive supranuclear palsy and multiple system atrophy. Brain (2008) 131:1362–72. doi: 10.1093/brain/awn065

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Seibl-Leven M, von Reeken C, Goldbrunner R, Grau S, Ruge MI, Galldiks N, et al. Clinical routine assessment of palliative care symptoms and concerns and caregiver burden in glioblastoma patients: an explorative field study. J Neurooncol. (2018) 138:321–33. doi: 10.1007/s11060-018-2800-1

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Harding R, Gao W, Jackson D, Pearson C, Murray J, Higginson IJ. Comparative analysis of informal caregiver burden in advanced cancer, dementia, and acquired brain injury. J Pain Symptom Manag. (2015) 50:445–52. doi: 10.1016/j.jpainsymman.2015.04.005

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Katsavos S, Artemiadis AK, Zacharis M, Argyrou P, Theotoka I, Chrysovitsanou C, et al. Predicting caregiving status and caregivers' burden in multiple sclerosis. A short report. Neurol Res. (2017) 39:13–5. doi: 10.1080/01616412.2016.1254942

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Geng D, Ou R, Miao X, Zhao L, Wei Q, Chen X, et al. Patients' self-perceived burden, caregivers' burden and quality of life for amyotrophic lateral sclerosis patients: a cross-sectional study. J Clin Nurs. (2016) 26:3188–99. doi: 10.1111/jocn.13667

PubMed Abstract | CrossRef Full Text | Google Scholar