A Dynamic Nomogram to Predict the Risk of Stroke in Emergency Department Patients With Acute Dizziness

Objective: To develop a risk prediction tool for acute ischemic stroke (AIS) for patients presenting to the emergency department (ED) with acute dizziness/vertigo or imbalance.

Method: A prospective, multicenter cohort study was designed, and adult patients presenting with dizziness/vertigo or imbalance within 14 days were consecutively enrolled from the EDs of 4 tertiary hospitals between August 10, 2020, and June 10, 2021. Stroke was diagnosed by CT or MRI performed within 14 days of symptom onset. Participants were followed-up for 30 days. The least absolute shrinkage and selection operator (LASSO) logistic regression analysis was conducted to extract predictive factors that best identified patients at high risk of stroke to establish a prediction model. Model discrimination and calibration were assessed and its prediction performance was compared with the age, blood pressure, clinical features, duration, and diabetes (ABCD2) score, nystagmus scheme, and finger to nose test.

Results: In this study, 790 out of 2,360 patients were enrolled {median age, 60.0 years [interquartile range (IQR), 51–68 years]; 354 (44.8%) men}, with complete follow-up data available. AIS was identified in 80 patients. An online web service tool (https://neuroby.shinyapps.io/dynnomapp/) was developed for stroke risk prediction, including the variables of sex, trigger, isolated symptom, nausea, history of brief dizziness, high blood pressure, finger to nose test, and tandem gait test. The model exhibited excellent discrimination with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.889 (95% CI: 0.855–0.923), compared with the ABCD2 score, nystagmus scheme, and finger to nose test [0.712 (95% CI, 0.652–0.771), 0.602 (95% CI, 0.556–0.648), and 61.7 (95% CI, 0.568–0.666) respectively].

Conclusion: Our new prediction model exhibited good performance and could be useful for stroke identification in patients presenting with dizziness, vertigo, or imbalance. Further externally validation study is needed to increase the strength of our findings.

Introduction

It has been established that stroke accounts for 2–13.4% of patients presenting to the emergency department (ED) with dizziness (1–5). Stroke is the second leading cause of death globally and has a limited treatment time window (6–8). Misdiagnosis can affect treatment decision-making, thus seriously impacting disease outcomes and patient quality of life, emphasizing the importance of timely stroke diagnosis in patients with dizziness (9–11). In this respect, it has been shown that over one-third of stroke cases are missed at the first visit by emergency physicians (EPs) (12). The misdiagnosis rate can be as high as 24–60%, especially when the symptoms are mild, non-specific, and transient (13).

Many efforts have been made to differentiate stroke from other causes of dizziness. The age, blood pressure, clinical features, duration, and diabetes (ABCD2) score with an area under the receiver operating characteristic (ROC) curve (AUC) of 0.79, and head impulse, nystagmus pattern, test of skew (HINTS) test with high sensitivity/specificity (100%/96%), are the two most widely acknowledged tools for stroke identification in patients with acute dizziness (14–17). The features of posterior circulation stroke, which had a triple misdiagnosis rate than that of anterior circulation stroke, are not included in ABCD2 score, which could probably decrease its diagnostic accuracy (18–21). Recently, a combination of ABCD2 ≥ 4 and a central pattern of nystagmus has been shown to yield higher sensitivity than the ABCD2 score alone for identifying stroke (22). Notwithstanding that the HINTS is reported to yield good diagnostic performance, studies report their low usage during clinical practice in EDs, as only 30% of EPs agree with the use of HINTS in patients with dizziness, not to mention the usage of head impulse, nystagmus pattern, test of skew, acute hearing loss (HINTS-plus) (23). Truncal ataxia, an easy-to-evaluate test, has been shown to yield high sensitivity to differentiate stroke from acute vestibular syndrome when combined with the nystagmus test (24). Posterior circulation ischemia (PCI) score, TriAGe+ score, STANDING algorithm, and DEFENSIVE scale were recently studied to estimate the risk of stroke in patients with dizziness (25–28). NLR, S100B, and NSE have been reported as new blood markers for the prediction of stroke in patients with dizziness (29–31). However, most of these prediction approaches were developed with small sample sizes, retrospectively collection, no validation, and have not been applied in clinical practice. Early risk stratification of dizzy patients is crucial, and misdiagnosis of stroke can lead to serious complications and poor outcomes. Thus, it is still challenging to discriminate stroke from patients with dizziness in the ED.

The present study sought to develop and validate a clinical prediction model based on easy-to-get predictors to identify patients with dizziness at risk of stroke in the ED.

Methods

Study Design and Setting

This was a multi-center, observational study that prospectively enrolled patients from August 10, 2020, to June 10, 2021. The study was conducted in the EDs at Wuhan Union Hospital, Wuhan Union Hospital West Campus, Wuhan Fourth Hospital (Puai Hospital), and Hubei Provincial Hospital of Integrated Chinese and Western Medicine. All study centers were teaching hospitals and home to the standardized resident training program. The four hospitals have a combined annual ED visit of ~200,000 patients, providing 24-h neurological consultation and emergency neuroimaging facilities.

The study was approved by the Ethics Committees of Tongji Medical College, Huazhong University of Science and Technology (S227) and was registered in the Chinese Clinical Trial Registry (www.chictr.org.cn, ChiCTR2000037496). The research was conducted in accordance with the Declaration of Helsinki and written informed consent was obtained from all participants.

Participants

Adult patients presenting with dizziness/vertigo or imbalance at the EDs were consecutively screened by 2 trained investigators via the ED triage electronic medical record system 24 h daily. Dizziness/vertigo and imbalance were defined in accordance with the consensus document of Bárány Society (32). Patients were excluded for (1) more than 14 days duration since symptom onset at admission, (2) chronic recurrent symptoms (defined as no less than 5 prior episodes similar in quality, intensity, and duration to the current symptoms, with at least one episode over a year and one within the last year), (3) history of multiple sclerosis, (4) dizziness thought to be the result of trauma, orthostatic hypotension, medication intoxication, hypoglycemia, or a known medical or neurologic disorder (e.g., hepatic encephalopathy and hydrocephalus), (5) central nervous system (CNS) examination abnormalities (e.g., hemiparesis, hemisensory loss, and gaze palsy) with a National Institutes of Health Stroke Scale (NIHSS) ≥5 [mild abnormalities (e.g., mild axial ataxia, dysarthria, or sensory symptoms) were not excluded], (6) patient refusal of neuroimaging tests or with contraindications to MRI or CT, or (7) refusal to participate.

The data of eligible patients were collected through detailed interviews by 2 investigators using a structured case report form customized for this study and blinded to neurologic exam abnormalities and primary outcome measures, such as CT and MRI.

Baseline Clinical Measurements

Baseline clinical variables, such as demographic characteristics, presenting symptoms, and medical history were obtained directly from the participants. A standardized physical examination was performed by two neurologists well trained in neuro-otology before the CT and/or MRI exam or blinded to the neuroimaging results. The physical examination included ocular motor examination, visual field assessment, assessment of cranial nerves, muscle strength, sensation, coordination, balance, Dix-Hallpike test, and hearing impairment by finger rub. During the nystagmus test, the eyes of patients were observed by investigators in primary position (straight-ahead) and then sequentially with gaze directed ~30–40 degrees left, right, up, and down. In each position, the vector of the fast-beating component and the intensity of nystagmus were recorded. In addition, the nystagmus examinations were recorded by video and estimated by a third investigator. A positive result, central pattern of nystagmus, was identified when any of bidirectional gaze-evoked nystagmus (GEN), persistent vertical nystagmus scored in any position, or isolated torsional nystagmus was observed (16). Bidirectional gaze-evoked nystagmus was defined as the presence of right-beat nystagmus when looking to the right and left-beat nystagmus when looking to the left, with the nystagmus lasted longer than 5 s. Any other nystagmus or no nystagmus were recognized as negative. During the finger to nose test, participants were asked to touch the tip of their nose with the index finger and repeatedly move the index finger to touch the finger of examiner and back to the nose (33). A positive result was determined if the subject tended to push the test finger upward, toward the ceiling and above the line between the finger of examiner and the nose of subject, showing dysmetria (34). A tandem gait test was conducted, and a positive test was observed when the patient could not walk 10 consecutive steps straight in the heel-to-toe manner with eyes open in up to 2 attempts (35–37). The ABCD2 score, NIHSS, modified Rankin Scale (mRS), and the Barthel index were calculated at the first visit.

From the electronic medical record system, we collected data on the arrival time, vital signs, such as body temperature, heart rate, and blood pressure, first laboratory tests of blood routine, kidney function, and coagulation parameters within 24 h after symptoms onset, and other routine laboratory parameters and administered medications by two investigators independently. Investigators were blinded to the neuroimaging findings and neurological exam abnormalities when they extracted the data.

Outcome Measures

The outcome was an imaging-based definition of AIS according to the international diagnostic guidelines. All enrolled patients completed either CT or MRI with diffusion-weighted imaging (DWI) and apparent diffusion coefficient of the brain within 14 days after symptoms onset. Besides, the MRI was performed more than 48 h from symptom onset to reduce the missed diagnosis due to false-negative findings (38, 39). Diffusion-weighted MRIs were performed with either a 1.5 or 3 T MRI machine and the slice thickness was 5 mm with a 1.5 mm slice gap. The original images from the 4 research centers were interpreted by two board-certified neuroradiologists blinded to the clinical information.

To rule out the possibility of missed stroke diagnosis and avoid verification bias, all patients enrolled were followed-up by a structured telephone interview a month after onset. Participants were asked whether they had undergone any new neuroimaging test after discharge and were requested to provide imaging results once they had. Moreover, we checked for revisits to the ED or hospitalization using the electronic medical record system.

Predictor Variables

We searched for predictors of stroke from keynote papers (3, 31, 40–50). Candidate predictors were identified when the factors were consistently reported in predictive research on patients with dizziness, easily ascertained, and routine diagnostic tests for patients with stroke. The candidate predictors employed for the development of a novel risk prediction model in our study are listed in Table 1. Dizziness/vertigo or imbalance with “trigger” was defined as symptoms presenting with an obvious trigger. Under most circumstances, a reproducible and repetitive relationship between the trigger stimulus and dizziness spell should be present. “Isolated symptom” was defined as dizziness/vertigo or imbalance without any other neurologic findings. “History of brief dizziness” was defined as a brief episode of dizziness that occurred within 3 months before the attack except for chronic recurrent symptoms. “Sleep disorder” was defined as sleep difficulty symptoms reported by the patient himself. High blood pressure was defined as systolic blood pressure (SBP) ≥ 140 mm Hg or diastolic blood pressure (DBP) ≥ 90 mm Hg after arriving at the ED.

TABLE 1

Table 1. Candidate predictors in this study.

Statistical Analysis

Data analysis was conducted using R (version 4.0.4; www.r-project.org) and SPSS (version 23.0; IBM, Chicago, IL, USA). Multiple imputation was used to deal with missing data. Values of missing variates (NLR, Hct, hsCRP, and uric acid) were imputed by assuming that data were missing at random with 50 imputations. All candidate predictor variables and several relevant variables were included in the imputation model. Predictive mean matching and logistic regression were applied to impute continuous and binary variables, respectively. The normality test was performed using the Shapiro–Wilk test. Continuous variables with a normal distribution were reported as mean ± SD, and those with a non-normal distribution were reported as median (interquartile range, IQR). Categorical variables were presented as numbers and percentages. The t-test or Mann–Whitney U-test was conducted to compare continuous variables. Comparisons of categorical variables were performed using Fisher's exact test.

Model Development

For the model development, the least absolute shrinkage and selection operator (LASSO) logistic regression was conducted to select variables that best identified patients at high risk of stroke through the selection of tuning parameter lambda (λ) (51, 52). The LASSO penalization factor selected predictors by shrinking coefficients for weaker predictors toward zero. The variables with a non-zero coefficient were chosen as stroke risk predictors. To avoid overfitting models, a 10-fold cross-validation was used for λ selection. Then, the multivariate logistic regression was used to compute the adjusted odds ratios (ORs) of the chosen predictors.

A nomogram was built to provide visualized risk prediction based on the multivariable analysis results using the “rms” R package. In the nomogram, the predicted probabilities were mapped into points on a scale from 0 to 100 in a user-friendly way. The point system works by ranking the effect estimates, regardless of the statistical significance, and it is influenced by the presence of other covariates in this model. The effect with the highest β (absolute value) will be assigned 100 points on the scale, and the remaining variables are assigned a smaller number of points proportional to their effect size (53). The total points accumulated by all the selected variables correspond to the predicted probability for a patient (54). Therefore, nomograms provide personalized predictions based on the specific clinical parameters of patient through an easy-to-use manner. Finally, to facilitate individualized prediction in clinical practice, we constructed a web-based calculator, written in R code using the Shiny framework (http://www.shinyapps.io/).

Model Evaluation

To evaluate the stability of the model and to perform model validation, a bootstrap resampling approach was used with 1,000 iterations. Calibration of the models was evaluated using Hosmer–Lemeshow goodness-of-fit test and a calibration curve was plotted to compare the agreement between the actual stroke rates and the model predicted stroke probabilities. A 45 degrees diagonal reference line denoted the line of perfect calibration, whereas deviation above or below this line reflects underprediction or overprediction. The discrimination power of the model was assessed through an ROC curve. The AUC and Harrell's concordance index (C-index) were measured to assess the prediction performance of the model. AUCs for the prediction model and other existing methods of risk prediction were compared using the DeLong's test. To determine the clinical utility of the model, decision curve analysis (DCA) was performed to quantify the net benefits at different threshold probabilities. This method is based on the principle that the relative harm of false positives and false negatives can be represented by probability threshold. The net benefit is obtained by subtracting the proportion of patients showing false positive results from the proportion showing true positive results, and then weighing the relative harm of false positive and false negative results (55). A good model usually yields a high net benefit over a wide range of threshold probabilities in the DCA curve. The following R packages were mainly used for our analysis: “glmnet” for LASSO Logistic regression algorithm; “pROC” for ROC analyses and DeLong's test; “rms” for calibration curve; and “rmda” for decision curve analysis. A two-sided p < 0.05 was considered statistically significant.

The present study was conducted following the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines (56).

Results

Patient Characteristics

In this study, 790 out of 2,360 patients who visited the EDs with dizziness/vertigo or imbalance were finally included. A total of 1,570 patients were excluded for the following reasons: time from symptom onset to admission longer than 14 days (n = 652), chronic recurrent symptoms (n = 519), history of multiple sclerosis (n = 3), patients with a history of trauma, orthostatic hypotension, medication intoxication, hypoglycemia, or a known medical or neurologic disorder (n = 316), CNS abnormalities with an NIHSS higher than 5 (n = 48), patient refusal or contraindicated for neuroimaging test (n = 17), and refusal to participate (n = 15) (Figure 1). All patients who met the inclusion criteria underwent neuroimaging; 628 (79.5%) underwent CT, and 456 (57.7%) underwent MRI. Data for 30-day follow-ups were available for all participants. Baseline clinical characteristics of the study population are presented in Table 2. A stroke was diagnosed in 80 patients (10.1%), and 59 (73.8%) were related to posterior circulation ischemia.

FIGURE 1

Figure 1. Flowchart of patient enrollment.

TABLE 2

Table 2. Demographic and clinical characteristics of 790 patients with dizziness/vertigo or imbalance.

Prediction Model Building

The results of LASSO analysis and cross-validation illustrated that the optimal value of λ was 0.0201005, with log (λ) = −3.907011, by applying the 1 SE of the minimum criteria (the 1-se criteria) (Figure S1). The multivariate Logistic regression analysis was then performed to establish a prediction model (Table S1). According to the results, 8 out of 20 potential predictors were screened out with non-zero coefficients for model construction. The variables, including sex, trigger, isolated symptom, nausea, history of brief dizziness, high blood pressure, finger to nose test, and tandem gait test, were then incorporated into a nomogram (Figure 2). To use the nomogram, the value of an individual participant is located on each variable axis, and a line is drawn straight upward to the Points axis to determine the number of points received for each variable value, add the points from all the variables. Finally, draw a line straight down from the Total Points axis to determine the Probability of stroke at the lower line of the nomogram. For convenient use during the clinical practice, we developed a web-based calculator (https://neuroby.shinyapps.io/dynnomapp/) based on the nomogram (Figure S2).

FIGURE 2

Figure 2. Nomogram to determine the probability of stroke in dizziness/vertigo or imbalance. To use the nomogram, users should locate the sex of a patient on the Sex axis first, and then draw a line straight upward to the Points axis to determine the number of points received for his sex. Repeat this process for all axes of the variables, sum the points achieved for each predictor, and located this sum on Total Points axis. Finally, draw a line straight down from the Total Points axis to determine the Probability of stroke at the lower line of the nomogram.

Prediction Model Performance Assessment

The prediction accuracy of the nomogram was estimated using the Hosmer–Lemeshow goodness-of-fit test yielding a p of 0.769, suggesting good prediction accuracy. The calibration plot illustrated in Figure S3 revealed a good consistency between the predicted probability and observed frequency of stroke. To assess the ability of model to discriminate patients with different diagnoses, we calculated the C-statistic. The unadjusted C-statistic of the model was 0.889, and the bootstrap-corrected C index was 0.876, demonstrating excellent discrimination. To compare the predicted value of our model with other established clinical methods, the ROC curves and AUCs of the ABCD2 score, finger to nose test, and nystagmus scheme for stroke prediction in patients with dizziness/vertigo or imbalance were plotted as shown in Figure 3, including an ROC curve of the newly-established model. The results suggested that our nomogram exhibited superior predictive performance with an AUC (95% CI) of 0.889 (95% CI: 0.855–0.923). The finger to nose test, which was a component of the nomogram, yielded a lower AUC value (0.617, 95% CI: 0.568–0.666) than our nomogram. The AUC values of the ABCD2 score and nystagmus scheme were 0.712 (95% CI: 0.652–0.771) and 0.602 (95% CI: 0.556–0.648).

FIGURE 3

Figure 3. Receiver operating characteristic (ROC) curves of the nomogram, age, blood pressure, clinical features, duration, and diabetes (ABCD2) score, finger to nose test, and nystagmus scheme for stroke prediction at emergency department (ED). Compared with the ABCD2 score, finger to nose test alone, and nystagmus scheme, the nomogram prediction model shows excellent diagnostic efficacy with especially high area under the ROC curve (AUC).

Clinical Utility

The DCA was applied to evaluate the clinical usefulness of the model. As illustrated in Figure 4, patients with dizziness/vertigo or imbalance would gain clinical benefit from our model if the risk threshold is <79%. For example, if the high-risk threshold probability of a patient is 30%, then the net benefit is 0.364 when we use the model to predict stroke risk.

FIGURE 4

Figure 4. A decision curve analysis (DCA) for the nomogram model to predict stroke. The x-axis indicates the threshold probability and the y-axis indicates the net benefit. The thick black line represents the net benefit when assume that no patient suffering stroke. The thin gray line shows the net benefit when assume that all patients suffering stroke. As shown by the red curve in the figure, when the threshold probability is <0.79, using our nomogram prediction model yields significant net clinical benefits to patients.

Discussion

We developed a novel practical model to predict the risk of stroke in patients presented to the ED with dizziness/vertigo or imbalance. The model was internally validated and exhibited good discrimination and calibration power. The model incorporated eight components: sex, trigger, isolated symptom, nausea, history of brief dizziness, high blood pressure, finger to nose test, and tandem gait test. In addition, we developed a web-based calculator for efficient and individualized prediction, which could assist physicians during clinical decision-making. To the best of our knowledge, this is the first study to construct a nomogram and web-based calculator for predicting the stroke risk of individual patients with dizziness.

Although many studies have been sought to explore novel approaches for early stroke recognition in patients with dizziness, few have been translated into the clinical practice. During the clinical practice, it has been established that misdiagnosis rates are high leading to treatment delay, especially in patients with posterior circulation stroke who exhibit non-specific symptoms and rarely present with unilateral weakness (57–59). It has been reported that 30% of unidentified stroke patients exhibited limb ataxia in EDs (60). Compared with the ABCD2 score, our nomogram model yielded a better diagnostic performance with higher AUC value. We hypothesize that the inclusion of the finger to nose and tandem gait tests might partly account for high accuracy of our model to a certain extent since both are reportedly powerful predictors for posterior circulation ischemia and minor stroke (35, 40, 41). In comparison, ABCD2 score was initially generated to assess the risk of early stroke after a transient ischemic attack and incorporate signs of anterior circulation abnormal and features of medical history (61). As for the HINTS test, the complex oculomotor examination might be the main reason limiting its use (13, 62). In addition, it is hard for patients to corporate to complete the examination, especially when they have persistent vertigo with serious concomitant symptoms, such as nausea, vomit, and postural instability. Unlike the HINTS and the nystagmus scheme, which are challenging for EPs to distinguish, the two tests included in our nomogram were common physical examinations that are easy to implement in the ED (62, 63). Besides, the HINTS test and nystagmus scheme were originally developed and used in patients with acute vestibular syndrome, however, they are not applicable in all patients with dizziness, limiting their clinical utility (64). Besides, skew deviation and head-impulse tests are the key findings for the HINTs. The nystagmus scheme alone will decrease its discrimination of risk prediction. Indeed, the proportion of patients showing pure vertical, torsional, GEN is rather low. These findings may explain the low AUC value associated with the nystagmus scheme in our study. The inclusion and exclusion criteria of the present research determined that the prediction model would have a wide application for patients with dizziness/vertigo or imbalance in clinical practice in the ED.

The acronym TiTrATE (timing, triggers, and targeted bedside eye examinations) is useful for targeting the examinations to help physicians make a specific diagnosis for patients with dizziness (65). The STANDING algorithm is a structured diagnostic algorithm that combined the presence and type of nystagmus, nystagmus direction, head impulse test, and evaluation of the standing position and gait into a four-step sequence (26). However, they are more suitable for use in dizziness clinics rather than EDs which need rapid assessment and triage treatment. The TriAGe+ Score and PCI score are two risk scores for stroke diagnosis in patients with dizziness, of which some components are the same as the variables in our prediction model. The TriAGe+ Score is composed of 8 variables, including triggers, atrial fibrillation, male gender, blood pressure ≥ 140/90 mm Hg, brainstem or cerebellar dysfunction, focal weakness or speech impairment, dizziness, and no history of vertigo or dizziness or labyrinth or vestibular disease, which were derived from a multivariate logistic regression analysis (27). A high sensitivity (96.6%) with a good negative likelihood ratio (0.15) was identified when the cutoff value was determined as 5 points, outperformed the ABCD2 score for stroke prediction. The reason for the failure of the score to be applied clinically may be due to the retrospectively data collection, single center study, and no model validation. The PCI score was established with a high AUC of 0.82 whereas the study patients were retrieved from the department of neurology retrospectively (25). It is not well suited for patients in ED. In comparison, our prediction model, prospectively collected dizziness/vertigo or imbalance patients from 4 EDs, shows high discrimination performance and good prediction accuracy, will be a useful tool for Eps.

The nomogram prediction model incorporated eight variables. Consisting with the literature, the variables of “isolated symptom” and “history of brief dizziness” were inversely associated with stroke onset while “male” and “high blood pressure” on admission were associated with a high risk of stroke in our study (44, 45, 66, 67). “Nausea” and “tandem gait test” were selected as predictors in a stroke risk stratification method for the first time. The “finger to nose test” is an easy-to-perform test for EP and has been proved to be a good predictor of stroke. In fact, addition of the finger to nose test to the emergency medical services was shown to improve stroke recognition (40). Above all, variables incorporated into the nomogram are readily accessible and do not require laboratory tests or instrumental examinations, making the nomogram a simple-to-use prediction model. Furthermore, our developed online web service prediction model can be a powerful and convenient tool for EPs to evaluate the stroke risk in patients with dizziness/ vertigo or imbalance.

Strength and Limitation

The main strength of our study is that this study was a prospective multicenter observational study that enrolled participants from four different tertiary hospitals. Moreover, predictors incorporated in our nomogram are well defined and readily available. Then, we incorporated the prediction model into an online risk assessment calculator that could be convenient for clinical practice. Besides, our nomogram yielded significantly higher AUC values than previous models reported in the literature.

Nonetheless, the present study has some limitations. First of all, although our model was internally validated by bootstrap resampling to demonstrate its applicability, external validation is essential to increase the robustness of our results. Furthermore, not all participants underwent DWI. Importantly, CT and a month of follow-up were conducted based on clinical guidelines to ensure that stroke cases were not missed, which could have significantly minimized the risk of misclassification bias (68). Besides, blood routine, hsCRP and uric acid were not available for all patients and were usually available in patients at a high risk of stroke. Missing values of these variables were imputed for analysis to enhance the precision of the established model. LASSO is generally considered superior to logistic regression since the predictive model constructed by LASSO is more stable and handles the problem of correlated inputs. However, only a limited number of features can be selected, and model interpretability is low in cases of low dimension. Finally, the number of stroke events associated with each predictor was small and combined with the relatively limited sample size, might restrict the applicability of the model.

Conclusion

Our dynamic nomogram, which incorporates clinical data, symptoms, and physical examination, has significant value for estimating the probability of stroke in patients presenting to the ED with dizziness. If future validation is conducted, this tool may have huge prospects for helping EPs differentiate stroke from non-stroke dizziness and assist in therapeutic decision-making during clinical practice.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology (S227). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

YB contributed to the concept and design, data analysis and interpretation, and writing manuscript. FC performed data interpretation, writing, and final approval of the manuscript. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We would like to thank all the nursing staff in the EDs and thank Dr. Zhou Sun, Dr. Yuran Zhou (Wuhan Union hospital), Dr. Jing Liu (Wuhan Fourth Hospital), and Dr. Kun Li (Hubei Provincial Hospital of Integrated Chinese and Western Medicine) for their participation and collection of data. We thank all patients involved in this study.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fneur.2022.839042/full#supplementary-material

Abbreviations

CI, confidence interval; ED, emergency department; EP, emergency physician; ROC, receiver operating characteristic curve; AUC, area under the receiver operating characteristics curve; AIS, acute ischemic stroke; NIHSS, National Institutes of Health Stroke Scale; MRI, magnetic resonance imaging; CT, computed tomography; mRS, modified Rankin Scale; DWI, diffusion-weighted imaging; LASSO, the least absolute shrinkage and selection operator; GEN, gaze-evoked nystagmus.

References

1. Kerber KA, Brown DL, Lisabeth LD, Smith MA, Morgenstern LB. Stroke among patients with dizziness, vertigo, and imbalance in the emergency department: a population-based study. Stroke. (2006) 37:2484–7. doi: 10.1161/01.STR.0000240329.48263.0d

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Zwergal A, Mohwald K, Salazar Lopez E, Hadzhikolev H, Brandt T, Jahn K, et al. A prospective analysis of lesion-symptom relationships in acute vestibular and ocular motor stroke. Front Neurol. (2020) 11:822. doi: 10.3389/fneur.2020.00822

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Chase M, Goldstein JN, Selim MH, Pallin DJ, Camacho MA, O'Connor JL, et al. A prospective pilot study of predictors of acute stroke in emergency department patients with dizziness. Mayo Clin Proc. (2014) 89:173–80. doi: 10.1016/j.mayocp.2013.10.026

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Newman-Toker DE, Hsieh YH, Camargo CA Jr, Pelletier AJ, Butchy GT, et al. Spectrum of dizziness visits to US emergency departments: cross-sectional analysis from a nationally representative sample. Mayo Clin Proc. (2008) 83:765–75. doi: 10.4065/83.7.765

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Goeldlin M, Gaschen J, Kammer C, Comolli L, Bernasconi CA, Spiegel R, et al. Frequency, aetiology, and impact of vestibular symptoms in the emergency department: a neglected red flag. J Neurol. (2019) 266:3076–86. doi: 10.1007/s00415-019-09525-4

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Wu S, Wu B, Liu M, Chen Z, Wang W, Anderson CS, et al. Stroke in China: advances and challenges in epidemiology, prevention, and management. Lancet Neurol. (2019) 18:394–405. doi: 10.1016/S1474-4422(18)30500-3

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Global regional and and national age-sex-specific mortality for 282 causes of death in 195 countries and territories 1980-2017: 1980-2017: a systematic analysis for the global burden of disease study 2017. Lancet. (2018) 392:1736–88. doi: 10.1016/S0140-6736(18)32203-7

PubMed Abstract | CrossRef Full Text

8. Powers WJ, Rabinstein AA, Ackerson T, Adeoye OM, Bambakidis NC, Becker K, et al. 2018 Guidelines for the early Management of Patients With Acute Ischemic Stroke. A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association. Stroke. (2018) 49:e46–110. doi: 10.1161/STR.0000000000000158

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Pfieffer ML, Anthamatten A, Glassford M. Assessment and treatment of dizziness and vertigo. Nurse Pract. (2019) 44:29–36. doi: 10.1097/01.NPR.0000579744.73514.4b

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Saber Tehrani AS, Kattah JC, Kerber KA, Gold DR, Zee DS, Urrutia VC, et al. Diagnosing stroke in acute dizziness and vertigo: pitfalls and pearls. Stroke. (2018) 49:788–95. doi: 10.1161/STROKEAHA.117.016979

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Newman-Toker DE. Missed stroke in acute vertigo and dizziness: it is time for action, not debate. Ann Neurol. (2016) 79:27–31. doi: 10.1002/ana.24532

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Moeller JJ, Kurniawan J, Gubitz GJ, Ross JA, Bhan V. Diagnostic accuracy of neurological problems in the emergency department. Can J Neurol Sci. (2008) 35:335–41. doi: 10.1017/S0317167100008921

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Tarnutzer AA, Lee SH, Robinson KA, Wang Z, Edlow JA, Newman-Toker DE. ED misdiagnosis of cerebrovascular events in the era of modern neuroimaging: a meta-analysis. Neurology. (2017) 88:1468–77. doi: 10.1212/WNL.0000000000003814

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Navi BB, Kamel H, Shah MP, Grossman AW, Wong C, Poisson SN, et al. Application of the ABCD2 score to identify cerebrovascular causes of dizziness in the emergency department. Stroke. (2012) 43:1484–9. doi: 10.1161/STROKEAHA.111.646414

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Kattah JC, Talkad AV, Wang DZ, Hsieh YH, Newman-Toker DE. HINTS to diagnose stroke in the acute vestibular syndrome: three-step bedside oculomotor examination more sensitive than early MRI diffusion-weighted imaging. Stroke. (2009) 40:3504–10. doi: 10.1161/STROKEAHA.109.551234

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Kerber KA, Meurer WJ, Brown DL, Burke JF, Hofer TP, Tsodikov A, et al. Stroke risk stratification in acute dizziness presentations: a prospective imaging-based study. Neurology. (2015) 85:1869–78. doi: 10.1212/WNL.0000000000002141

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Spiegel R, Kirsch M, Rosin C, Rust H, Baumann T, Sutter R, et al. Dizziness in the emergency department: an update on diagnosis. Swiss Med Wkly. (2017) 147:w14565. doi: 10.4414/smw.2017.14565

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Venkat A, Cappelen-Smith C, Askar S, Thomas PR, Bhaskar S, Tam A, et al. Factors associated with stroke misdiagnosis in the emergency department: a retrospective case-control study. Neuroepidemiology. (2018) 51:123–7. doi: 10.1159/000491635

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Arch AE, Weisman DC, Coca S, Nystrom KV, Wira CR 3rd, Schindler JL. Missed ischemic stroke diagnosis in the emergency department by emergency medicine and neurology services. Stroke. (2016) 47:668–73. doi: 10.1161/STROKEAHA.115.010613

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Calic Z, Cappelen-Smith C, Anderson CS, Xuan W, Cordato DJ. Cerebellar infarction and factors associated with delayed presentation and misdiagnosis. Cerebrovasc Dis. (2016) 42:476–84. doi: 10.1159/000448899

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Lee H, Sohn SI, Cho YW, Lee SR, Ahn BH, Park BR, et al. Cerebellar infarction presenting isolated vertigo: frequency and vascular topographical patterns. Neurology. (2006) 67:1178–83. doi: 10.1212/01.wnl.0000238500.02302.b4

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Wang W, Zhang Y, Pan Q, Liu J, Zhu Y, Tan G, et al. Central nystagmus plus ABCD identifying stroke in acute dizziness presentations. Acad Emerg Med. (2021) 28:1118–23. doi: 10.1111/acem.14295

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Kene MV, Ballard DW, Vinson DR, Rauchwerger AS, Iskin HR, Kim AS. Emergency physician attitudes, preferences, and risk tolerance for stroke as a potential cause of dizziness symptoms. West J Emerg Med. (2015) 16:768–76. doi: 10.5811/westjem.2015.7.26158

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Carmona S, Martinez C, Zalazar G, Moro M, Batuecas-Caletrio A, Luis L, et al. The diagnostic accuracy of truncal ataxia and HINTS as cardinal signs for acute vestibular syndrome. Front Neurol. (2016) 7:125. doi: 10.3389/fneur.2016.00125

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Chen R, Su R, Deng M, Liu J, Hu Q, Song Z. A posterior circulation ischemia risk score system to assist the diagnosis of dizziness. J Stroke Cerebrovasc Dis. (2018) 27:506–12. doi: 10.1016/j.jstrokecerebrovasdis.2017.09.027

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Vanni S, Pecci R, Edlow JA, Nazerian P, Santimone R, Pepe G, et al. Differential diagnosis of vertigo in the emergency department: a prospective validation study of the STANDING algorithm. Front Neurol. (2017) 8:590. doi: 10.3389/fneur.2017.00590

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kuroda R, Nakada T, Ojima T, Serizawa M, Imai N, Yagi N, et al. The TriAGe+ score for vertigo or dizziness: a diagnostic model for stroke in the emergency department. J Stroke Cerebrovasc Dis. (2017) 26:1144–53. doi: 10.1016/j.jstrokecerebrovasdis.2017.01.009

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Yamada S, Yasui K, Kawakami Y, Hasegawa Y, Katsuno M. DEFENSIVE stroke scale: novel diagnostic tool for predicting posterior circulation infarction in the emergency department. J Stroke Cerebrovasc Dis. (2019) 28:1561–70. doi: 10.1016/j.jstrokecerebrovasdis.2019.03.005

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kartal AG, Yilmaz S, Yaka E, Pekdemir M, Sarisoy HT, Cekmen MB, et al. Diagnostic value of S100B protein in the differential diagnosis of acute vertigo in the emergency department. Acad Emerg Med. (2014) 21:736–41. doi: 10.1111/acem.12420

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Sohn JH, Kim CH, Lee SH, Kim JH, Lee JJ. Diagnostic value of serum biomarkers for differentiating central and peripheral causes of acute vertigo. Front Med. (2020) 7:84. doi: 10.3389/fmed.2020.00084

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Lee SH, Yun SJ, Ryu S, Choi SW, Kim HJ, Kang TK, et al. Utility of neutrophil-to-lymphocyte ratio (NLR) as a predictor of acute infarction in new-onset acute vertigo patients without neurologic and computed tomography abnormalities. J Emerg Med. (2018) 54:607–14. doi: 10.1016/j.jemermed.2017.12.058

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Bisdorff A, Von Brevern M, Lempert T, Newman-Toker DE. Classification of vestibular symptoms: towards an international classification of vestibular disorders. J Vestib Res. (2009) 19:1–13. doi: 10.3233/VES-2009-0343

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dominguez-Vega ZT, Dubber D, Elting JWJ, Sival DA, Maurits NM. Instrumented classification of patients with early onset ataxia or developmental coordination disorder and healthy control children combining information from three upper limb SARA tests. Eur J Paediatr Neurol. (2021) 34:74–83. doi: 10.1016/j.ejpn.2021.07.009

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Power L, Pathirana P, Horne M, Milne S, Marriott A, Szmulewicz DJ. Instrumented objective clinical examination of cerebellar ataxia: the upper and lower limb-a review. Cerebellum. (2021). doi: 10.1007/s12311-021-01253-8. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Margolesky J, Singer C. How tandem gait stumbled into the neurological exam: a review. Neurol Sci. (2018) 39:23–9. doi: 10.1007/s10072-017-3108-1

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Yoo D, Kang KC, Lee JH, Lee KY, Hwang IU. Diagnostic usefulness of 10-step tandem gait test for the patient with degenerative cervical myelopathy. Sci Rep. (2021) 11:17212. doi: 10.1038/s41598-021-96725-6

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Gray M, Wilson JC, Potter M, Provance AJ, Howell DR. Female adolescents demonstrate greater oculomotor and vestibular dysfunction than male adolescents following concussion. Phys Ther Sport. (2020) 42:68–74. doi: 10.1016/j.ptsp.2020.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Saber Tehrani AS, Kattah JC, Mantokoudis G, Pula JH, Nair D, Blitz A, et al. Small strokes causing severe vertigo: frequency of false-negative MRIs and nonlacunar mechanisms. Neurology. (2014) 83:169–73. doi: 10.1212/WNL.0000000000000573

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kabra R, Robbie H, Connor SE. Diagnostic yield and impact of MRI for acute ischaemic stroke in patients presenting with dizziness and vertigo. Clin Radiol. (2015) 70:736–42. doi: 10.1016/j.crad.2015.01.016

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Oostema JA, Chassee T, Baer W, Edberg A, Reeves MJ. Educating paramedics on the finger-to-nose test improves recognition of posterior stroke. Stroke. (2019) 50:2941–3. doi: 10.1161/STROKEAHA.119.026221

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Li N, Li J, Gao T, Wang D, Du Y, Zhao X. Gait and balance disorder in patients with transient ischemic attack or minor stroke. Neuropsychiatr Dis Treat. (2021) 17:305–14. doi: 10.2147/NDT.S289158

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Kim BJ, Jang SK, Kim YH, Lee EJ, Chang JY, Kwon SU, et al. Diagnosis of acute central dizziness with simple clinical information using machine learning. Front Neurol. (2021) 12:691057. doi: 10.3389/fneur.2021.691057

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Lee CC, Su YC, Ho HC, Hung SK, Lee MS, Chou P, et al. Risk of stroke in patients hospitalized for isolated vertigo: a four-year follow-up study. Stroke. (2011) 42:48–52. doi: 10.1161/STROKEAHA.110.597070

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Kim Y, Faysel M, Balucani C, Yu D, Gilles N, Levine SR. Ischemic stroke predictors in patients presenting with dizziness, imbalance, and vertigo. J Stroke Cerebrovasc Dis. (2018) 27:3419–24. doi: 10.1016/j.jstrokecerebrovasdis.2018.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Navi BB, Kamel H, Shah MP, Grossman AW, Wong C, Poisson SN, et al. Rate and predictors of serious neurologic causes of dizziness in the emergency department. Mayo Clin Proc. (2012) 87:1080–8. doi: 10.1016/j.mayocp.2012.05.023

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Akoglu EU, Akoglu H, Cimilli Ozturk T, Onur B, Eroglu SE, Onur O, et al. Predictors of false negative diffusion-weighted MRI in clinically suspected central cause of vertigo. Am J Emerg Med. (2018) 36:615–9. doi: 10.1016/j.ajem.2017.09.038

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Cheung CS, Mak PS, Manley KV, Lam JM, Tsang AY, Chan HM, et al. Predictors of important neurological causes of dizziness among patients presenting to the emergency department. Emerg Med J. (2010) 27:517–21. doi: 10.1136/emj.2009.078014

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Wang Y, Liu J, Cui Z, Yan L, Si J. Analysis of risk factors in patients with peripheral vertigo or central vertigo. Neurologist. (2018) 23:75–8. doi: 10.1097/NRL.0000000000000179

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Eryigit U, Altunayoglu Cakmak V, Sahin A, Tatli O, Pasli S, Gazioglu G, et al. The diagnostic value of the neutrophil-lymphocyte ratio in distinguishing between subarachnoid hemorrhage and migraine. Am J Emerg Med. (2017) 35:1276–80. doi: 10.1016/j.ajem.2017.03.063

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Lin F, Chen Y, Wan M, Chen W, Jia W. High-sensitivity C-reactive protein as an indicator of ischemic stroke in patients with isolated acute vestibular syndrome: retrospective observational study. Medicine. (2019) 98:e18097. doi: 10.1097/MD.0000000000018097

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Software. (2010) 33:1–22. doi: 10.18637/jss.v033.i01

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Qian W, Yang Y. Model selection via standard error adjusted adaptive lasso. Ann Inst Stat Math. (2013) 65:295–318. doi: 10.1007/s10463-012-0370-0

CrossRef Full Text | Google Scholar

53. Iasonos A, Schrag D, Raj GV, Panageas KS. How to build and interpret a nomogram for cancer prognosis. J Clin Oncol. (2008) 26:1364–70. doi: 10.1200/JCO.2007.12.9791

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Stephenson AJ, Scardino PT, Eastham JA, Bianco FJ Jr., Dotan ZA, et al. Postoperative nomogram predicting the 10-year probability of prostate cancer recurrence after radical prostatectomy. J Clin Oncol. (2005) 23:7005–12. doi: 10.1200/JCO.2005.01.867

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. (2006) 26:565–74. doi: 10.1177/0272989X06295361

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Collins GS, Reitsma JB, Altman DG, Moons KGM, Grp T. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Eur Urol. (2015) 67:1142–51. doi: 10.1016/j.eururo.2014.11.025

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Madsen TE, Khoury J, Cadena R, Adeoye O, Alwell KA, Moomaw CJ, et al. Potentially missed diagnosis of ischemic stroke in the emergency department in the greater cincinnati/northern kentucky stroke study. Acad Emerg Med. (2016) 23:1128–35. doi: 10.1111/acem.13029

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Sarraj A, Medrek S, Albright K, Martin-Schild S, Bibars W, Vahidy F, et al. Posterior circulation stroke is associated with prolonged door-to-needle time. Int J Stroke. (2015) 10:672–8. doi: 10.1111/j.1747-4949.2012.00952.x

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Liberman AL, Prabhakaran S. Stroke chameleons and stroke mimics in the emergency department. Curr Neurol Neurosci Rep. (2017) 17:15. doi: 10.1007/s11910-017-0727-0

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Oostema JA, Konen J, Chassee T, Nasiri M, Reeves MJ. Clinical predictors of accurate prehospital stroke recognition. Stroke. (2015) 46:1513–7. doi: 10.1161/STROKEAHA.115.008650

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Johnston SC, Rothwell PM, Nguyen-Huynh MN, Giles MF, Elkins JS, Bernstein AL, et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet. (2007) 369:283–92. doi: 10.1016/S0140-6736(07)60150-0

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Ohle R, Montpellier RA, Marchadier V, Wharton A, McIsaac S, Anderson M, et al. Can emergency physicians accurately rule out a central cause of vertigo using the HINTS examination? A systematic review and meta-analysis. Acad Emerg Med. (2020) 27:887–96. doi: 10.1111/acem.13960

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Dumitrascu OM, Torbati S, Tighiouart M, Newman-Toker DE, Song SS. Pitfalls and rewards for implementing ocular motor testing in acute vestibular syndrome: a pilot project. Neurologist. (2017) 22:44–7. doi: 10.1097/NRL.0000000000000106

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Kattah JC. Update on HINTS plus, with discussion of pitfalls and pearls. J Neurol Phys Ther. (2019) 43 (Suppl. 2):S42–5. doi: 10.1097/NPT.0000000000000274

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Newman-Toker DE, Edlow JA. TiTrATE: a novel, evidence-based approach to diagnosing acute dizziness and vertigo. Neurol Clin. (2015) 33:577–99, viii. doi: 10.1016/j.ncl.2015.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Atzema CL, Grewal K, Lu H, Kapral MK, Kulkarni G, Austin PC. Outcomes among patients discharged from the emergency department with a diagnosis of peripheral vertigo. Ann Neurol. (2016) 79:32–41. doi: 10.1002/ana.24521

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Kerber KA, Zahuranec DB, Brown DL, Meurer WJ, Burke JF, Smith MA, et al. Stroke risk after nonstroke emergency department dizziness presentations: a population-based cohort study. Ann Neurol. (2014) 75:899–907. doi: 10.1002/ana.24172

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Reitsma JB, Rutjes AWS, Khan KS, Coomarasamy A, Bossuyt PM. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. J Clin Epidemiol. (2009) 62:797–806. doi: 10.1016/j.jclinepi.2009.02.005

PubMed Abstract | CrossRef Full Text | Google Scholar