An Umbrella Review With Meta-Analysis of Chest Computed Tomography for Diagnosis of COVID-19: Considerations for Trauma Patient Management

Background: RT-PCR testing is the standard for diagnosis of COVID-19, although it has its suboptimal sensitivity. Chest computed tomography (CT) has been proposed as an additional tool with diagnostic value, and several reports from primary and secondary studies that assessed its diagnostic accuracy are already available. To inform recommendations and practice regarding the use of chest CT in the in the trauma setting, we sought to identify, appraise, and summarize the available evidence on the diagnostic accuracy of chest CT for diagnosis of COVID-19, and its application in emergency trauma surgery patients; overcoming limitations of previous reports regarding chest CT accuracy and discussing important considerations regarding its role in this setting.

Methods: We conducted an umbrella review using Living Overview of Evidence platform for COVID-19, which performs regular automated searches in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and more than 30 other sources. The review was conducted following the JBI methodology for systematic reviews. The Grading of Recommendations, Assessment, Development, and Evaluation approach for grading the certainty of the evidence is reported (registered in International Prospective Register of Systematic Reviews, CRD42020198267).

Results: Thirty studies that fulfilled selection criteria were included; 19 primary studies provided estimates of sensitivity (0.91, 95%CI = [0.88–0.93]) and specificity (0.73, 95%CI = [0.61; 0.82]) of chest CT for COVID-19. No correlation was found between sensitivities and specificities (ρ = 0.22, IC95% [–0.33; 0.66]). Diagnostic odds ratio was estimated at: DOR = 27.5, 95%CI (14.7; 48.5). Evidence for sensitivity estimates was graded as MODERATE, and for specificity estimates it was graded as LOW.

Conclusion: The value of chest CT appears to be that of an additional screening tool that can easily detect PCR false negatives, which are reportedly highly frequent. Upon the absence of PCR testing and impossibility to perform RT-PCR in trauma patients, chest CT can serve as a substitute with increased value and easy implementation.

Systematic Review Registration: [www.crd.york.ac.uk/prospero], identifier [CRD42020198267].

Key messages

What is already known on the subject?

RT-PCR testing has disadvantages for detecting COVID-19 timely in a trauma surgery setting. Sample processing can be long and requires specific laboratory protocols, which usually delay test results; and sampling can be difficult during basic and advanced trauma life support. Chest CT has been proposed as an additional tool in the diagnosis of COVID-19, and several reports from primary and secondary studies that assessed its diagnostic accuracy have been published.

What are the new findings?

Chest CT is a highly sensitive tool to detect COVID-19. Chest CT specificity was lower than sensitivity. Great variation was present between studies due to differences in design, index test definition and reference standards.

How might these results affect future research or surgical practice?

Chest CT is valuable when PCR testing is absent or obtaining timely results is not possible. Positive findings on chest CT should prompt additional protective measures in aerosolizing procedures for medical staff and isolating measures for the patient.

As emergency trauma patients typically undergo localized or full body CT scanning, imaging of the lungs and its interpretation by a radiologist is not expected to increase costs significantly and can be implemented as a screening tool in that setting.

One important consideration are patients with trauma or polytrauma involving the chest, as lung contusions and hemo- or pneumo-thorax will affect the readability of chest CT for pneumonia.

Background

During the COVID-19 pandemic, healthcare facilities all over the world are challenged when caring for trauma patients for whom a history of typical symptoms, close contacts and even vaccination status is not available because of agitation, intoxication or unconsciousness due to injury and sedation. Timely detection of cases is crucial to prompt adequate isolation measures and use of personal protective equipment to protect medical staff and other patients. These measures should be applied preventively in every trauma patient with need for emergency surgery, but shortages and sparse resources limit the compliance of these COVID-19 preventive recommendations in low and middle income countries (LMICs); and are thus applied almost exclusively in confirmed cases (1–3). Diagnosis is done with reverse transcriptase polymerase chain reaction (RT-PCR) to identify genetic material of the virus in nasopharyngeal or oropharyngeal swabs.

RT-PCR testing has some disadvantages that become more important in the emergency surgery setting: sample processing can be long and requires specific laboratory protocols, which usually delays test results; and sampling can be difficult during basic and advanced trauma life support. RT-PCR has varying sensitivity among different sampling modes: 97.2% (95%CI: 90.3–99.7%) in sputum; 62.3% (95%CI: 54.5–69.6%) in saliva; and 73.3% (95%CI: 68.1–78.0%) in nasopharyngeal and throat swabs; with a pooled sensitivity estimated at 84.8%, 95% CI = [76.8%; 92.4%]) (4, 5). Of these, nasopharyngeal and throat swabs are the most commonly applied. For both, a considerable rate of false negative results has been reported and is to be expected: an initial false negative RT-PCR results was measured as high as 54% of the time (4, 6). False positives, on the contrary, are very unlikely because specificity has been reported at 98.9%, 95% CI = [97.4%; 99.8%]) with low variability (4). Chest CT has been proposed as an additional tool in the diagnosis of COVID-19, and several reports from primary and secondary studies that assessed its diagnostic accuracy have been published. Chest CT cannot detect asymptomatic carriers, of course, but could detect COVID-19 pneumonias that single RT-PCR can miss (if available).

The above factors make RT-PCR flawed and unpractical for COVID-19 detection in trauma patients, and open the door for a role of chest CT (6). The reported accuracy of chest CT for COVID-19 diagnosis varies substantially and the validity of primary studies is variable, being affected by poor adherence to reporting guidelines and high risk of bias (7). To inform recommendations and practice regarding the use of chest CT in the in the emergency trauma setting with the above factors in mind, we sought to identify, appraise, and summarize the available evidence on the diagnostic accuracy of chest CT for rapid diagnosis of COVID-19, and discuss important considerations for its use.

Methods

We conducted a broad evidence synthesis (umbrella review) to summarize the diagnostic accuracy of chest CT imaging to detect COVID-19, the respiratory disease caused by the SARS-CoV-2 virus. A protocol of this review following the PRISMA statement was registered in the International Prospective Registry of Systematic Reviews (PROSPERO; CRD42020198267) and published in JMIR Research Protocols (8). This review was conducted following the JBI methodology and the Cochrane Handbook for systematic reviews of diagnostic accuracy (9).

Study Selection

Selection was carried out in Covidence (Melbourne, Australia). Two independent reviewers examined titles and abstracts for eligibility. Full-text review verified fulfillment of selection criteria. All decisions taken during screening were documented and are outlined in this report with a list of excluded studies. Any disagreements that arose between the reviewers were solved by consensus. The results of the search are presented in a Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flow diagram (10).

Selection Criteria

Population of Interest

We considered studies that assessed chest CT imaging for diagnosis of COVID-19 pandemic in trauma patients. Given the likelihood that reports on this specific population were limited, we also included studies of any patients with clinical suspicion of COVID-19 in any procedural and in-hospital setting as the diagnostic accuracy for detection of COVID-19 was considered extrapolatable to most trauma patients.

Types of Studies

Systematic reviews of diagnostic test accuracy studies (DTA) and individual DTAs not included in systematic reviews were included. To be considered a DTA, a study had to include patients with a diagnostic equipoise: patients with and without COVID-19, to accurately measure both sensitivity and specificity. Only studies published in English, Spanish or French were considered. We included pre-print studies identified in our search, but no ongoing studies were searched or considered.

Index Test

For eligibility, a study had to report positive or negative findings of COVID-19 in chest CT imaging, or report findings on chest CT imaging according to described radiologic scales such as the CO-RADS classification or the consensus by the Radiological Society of North America (11, 12). Imaging analyses other than positive or negative for COVID-19 were not used for metanalysis but were considered to summarize valuable information on radiologic reporting of COVID-19 chest CT imaging.

Search Strategy

We conducted searches in the L⋅OVE (Living OVerview of Evidence) platform for COVID-19, a system that performs automated regular searches in PubMed, Embase, Cochrane Central Register of Controlled Trials (CENTRAL), and over thirty other sources (13). When compared to manual searches, this platform consistently identifies all the available studies associated with the terms of interest. It allows for a fast (automated) search that is easy to update – a crucial element given the urgent need to answer the research question rapidly and thoroughly.

The platform was consulted on March 15, 2021, using the entry: (1) Diagnostic – Imaging tests – Computed tomography – Population Filter: COVID-19. The full search strategy and terms used to identify papers in L⋅OVE are presented in our registered protocol. Search through cross-referencing was also carried out to identify additional references.

Assessment of Methodological Quality

Eligible studies were critically appraised by a reviewer and verified by a second reviewer using the QADAS-2 tool for diagnostic test accuracy studies, and the AMSTAR-2 tool for systematic reviews and meta-analyses (9). The results of the critical appraisal are reported narratively and were considered for discussion of results. All included studies underwent data extraction and appraisal. Studies that applied single RT-PCR as a reference standard were consider to have evident risk of bias for estimating diagnostic accuracy due to the likelihood of an initial false negative result in symptomatic patients. If RT-PCR was not repeated or accompanied by adequate follow-up that discards other etiologies, diagnostic accuracy of CT imaging was likely to be biased, particularly in the calculation of specificity. If a single RT-PCR is used as reference standard, false negatives that could be detected by chest CT are wrongfully considered as false positives of chest CT, gravely distorting actual test accuracy. This aspect was rigorously assessed in primary DTA studies and systematic reviews to avoid metanalysis of studies with high risk of bias that directly and considerably affects estimations of diagnostic accuracy.

Data Extraction

Data were extracted from the included studies by a reviewer and verified by a second reviewer using a data extraction tool developed by the authors. The data extracted included specific details about the study population, the index test and reference standard used, other sources of bias, frequency of true negatives, false positives, true positives and false negatives. Disagreements were solved by consensus.

Data Synthesis

A narrative-only summary of review findings was planned. Nonetheless, the included reviews were found to either have high risk of bias in its estimates due to acceptance of single-PCR testing as a reference standard for considering studies in metanalysis, or did not include several of the identified primary DTA studies. This indicated the need to conduct a quantitative synthesis as well, as considered and planned in the published review protocol.

Meta-Analyses

Estimates of pooled sensitivity and specificity as well as other diagnostic summary measures were obtained by use of the bivariate random-effect model methods (14). This approach is the standard for DTA meta-analyses when sources of heterogeneity additional to threshold effect are expected (as is the case with chest CT for COVID-19), and averaged or weighted univariate methods for metanalysis of sensitivity and specificity are discouraged (14, 15). A bivariate method was considered most adequate as it incorporates unexplained variability in the analysis. Variation in specificity or sensitivity measures between studies can be attributed to differences in index tests, reference standards, study populations and settings. When sensitivity and specificity derive from the cut-off of a scale, they have a negative correlation, which is considered with the bivariate method as well (14). With this method, reports with more precise estimates have more weight. A secondary (sensitivity) meta-analyses was performed considering only studies with low risk of bias. Results are presented with pooled estimates of sensitivity, specificity, diagnostic odds ratio, forest plots and summary receiver operator characteristics (SROC) curve. Meta-analyses were conducted in RStudio version 1.3.10 using the “mada” package (16). Likelihood ratios for positive (LR +) and negative (LR-) results were calculated from pooled accuracy estimates.

Index Tests and Reference Standards Considered for Meta-Analyses

A definition of positive or negative chest CT had to be provided for adequate extraction of true positives, false negatives, true negatives and false positives. Index tests definitions were considered adequate if standardized or derived from internationally accepted recommendations for chest CT interpretation in COVID-19 patients (11, 12). Some studies based their index test on the dichotomization of these and other proposed scales for chest CT classification of COVID-19-suspected cases. Studies that did not report a specific positive or negative definition of chest CT results were eligible for metanalysis if information for calculation of the same cut-off dichotomization was possible. A reference standard was considered adequate when consisting of: multiple or repeated PCR-testing; or a composite of epidemiological, clinical and PCR according to World Health Organization (WHO) recommendations, which usually included follow-up to determine COVID-19+ or –. Studies with repeated PCR testing applied only to patients with negative results was considered adequate as a false positive result is highly unlikely, and was considered for primary metanalysis.

Assessing Certainty in the Findings

The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach for grading the certainty of the evidence was followed. Grading the certainty of the evidence was not planned if adaptation from the identified reviews that used the GRADE approach was considered complete and adequate (17, 18). Although the GRADE handbook suggests to assess evidence quality of diagnostic accuracy based on its impact on outcomes (i.e., the aftermath consequences of misdiagnosing patients), the interest on diagnostic accuracy for this review is not related to COVID-19 outcomes, but rather on the impact it could have for hospital personnel and other patients, because of contagion risk (19). Thus, a described alternative for assessing quality of diagnostic tests evidence of the GRADE approach that focuses on diagnostic accuracy estimates was used instead of considering test accuracy as a surrogate of patient outcomes. Given that pre-test probability of COVID-19 in trauma patients is low, the effect of diagnostic accuracy was assessed with pre-test probabilities of 1 and 10%, reflecting the scenario in which the diagnostic test is to be applied. The certainty of the evidence is reported in a summary of findings (SoF) table and was considered for interpretation and discussion of findings.

Publication Bias

Adequate methods for assessing publication bias in reviews of diagnostic test accuracy studies have not been developed. Funnel plots to assess asymmetry are designed for use in reviews of randomized trials and should not be used with diagnostic accuracy studies. Some available methods to assess publication bias have low power in the presence of heterogeneity, which is expected in diagnostic reviews, and thus interpretation of statistical evidence for publication bias derived from funnel plot is not recommended, as it does not necessarily imply publication bias (20, 21). Coherently, statistical assessment of publication bias was not undertaken. Furthermore, publication bias is unlikely in diagnostic test accuracy as there are no “positive” or “negative” results that could increase or decreased likelihood of publication, respectively, as is the case in review of other designs. Particularly for this review, risk of publication bias is expected to be low due to its unlikelihood in diagnosis topics and because several pre-print studies were identified and considered.

Results

Search Results

The search strategy and cross-referencing searches yielded 395 results. After duplicate screening of title and abstract 70 references were selected for full text review of which 30 met selection criteria and were included: 11 systematic reviews (5, 22–31) and 19 primary diagnostic accuracy studies (12, 32–49). Reasons for exclusion were related to not having a DTA design: only COVID-19 patients; only abnormal CT scans; not reporting data to compute sensitivity and specificity; or using single RT-PCR as a reference standard in all patients. Figure 1 is a PRISMA flow diagram depicting the selection process.

FIGURE 1

Figure 1. PRISMA flow diagram of the studies selection process.

None of the identified studies assessed chest CT diagnostic accuracy specifically in trauma patients; all were conducted in the emergency setting in patients with suspected COVID-19. This was expected as low probability of infection in trauma patients makes such studies impractical. As stated in our previously published protocol, the available evidence on chest CT accuracy for diagnosis of COVID-19 was to be considered and summarized, considering the indirectness of the evidence and discussing important considerations for its extrapolation to the trauma setting. Studies conducted in the emergency department setting had only slight differences, but overall, patients underwent chest CT scan examination at admission that was compared to an adequate reference standard for COVID-19; chest CT scan analysis was made by experienced radiologist that were in most cases blinded to reference standard results and determined a positive chest CT scan when patients had an image compatible with or typical of COVID-19. In some studies, no dichotomous (+ or –) index test result was presented, and the results of CT image were described or classified with radiological scales such as the proposed CO-RADS classification or the consensus by the Radiological Society of North America (11, 12). Definition of the index test result as positive or negative was similar and consider comparable among studies. The characteristics of the primary DTA studies considered for synthesis are summarized in Table 1.

TABLE 1

Table 1. Characteristics of studies included in metanalysis.

Risk of Bias in Included Studies

All included studies were assessed for risk of bias. Tables 2, 3 display the results of the risk of bias assessment. For systematic reviews, the formulation of a specific review question, the search strategies and resources to ensure identification of all relevant studies, methods to minimize errors in data extraction, and methods to combine studies’ results were adequate in most reviews. Nonetheless, all included systematic reviews were found to have at least two items that suggested risk of bias. A frequent issue was the inclusion of primary studies that used single RT-PCR testing as reference standard for all patients, and/or inclusion of studies that only considered patients diagnosed with COVID-19 (hence impairing assessment of false positive, true negative rates, and meta-analyses; not a DTA design). A flawed reference standard, as is the case of single RT-PCR due to high false negative rate, leads to severely biased estimates of sensitivity and specificity, as discussed in the methods section. To include such reports in metanalysis was considered inadequate. Thus, results from systematic reviews that included DTA studies that used single PCR testing as a reference standard were not considered for synthesis; neither were those results from reviews that included studies of only COVID-19-confirmed patients. Four reviews used appropriate selection criteria based on an appropriate reference standard and inclusion of patients independent of that reference standard result (22, 24, 30). One lacked a risk of bias assessment for the included studies and only two specified that this process was performed in duplicate (22, 24, 30). One used a single database as a resource to identify studies and was conducted early in the pandemic, leading to missing studies that were published later (22). These findings pointed to the need to conduct a new metanalysis to overcome those limitations.

TABLE 2

Table 2. Risk of bias assessment for included systematic reviews.

TABLE 3

Table 3. Risk of bias assessment for included diagnostic test accuracy studies.

Regarding primary DTA studies, assessment of risk of bias is presented only for studies considered for metanalysis and is displayed in Table 3. All studies used an adequate reference standard (see methods). Nonetheless, some studies performed further testing and follow up only in patients with initial negative results; while patients that had symptoms and a positive PCR test were considered positive. This is due to the inherent properties of RT-PCR testing (low sensitivity and high specificity), which confer confidence in positive results but skepticism in negative results. This reference standard was also considered adequate. Two studies report being unable to repeat PCR testing for initially negative patients in 1/3 of their study sample. These studies are marked as “with some concerns” (SC) regarding risk of bias.

Metanalysis and Summary Receiver Operator Characteristics Estimation

Of the 19 primary DTA studies 15 were meta-analyzed. Four studies that reported CT findings according to a radiological scale without positive or negative result (index test) are summarized but were not considered in metanalysis (12, 33, 38, 40). Figure 2 displays forest plot of sensitivities and specificities reported by the primary studies; pooled estimates are derived from bivariate metanalysis. Sensitivity of chest CT for COVID-19 was estimated at 0.91, 95%CI = (0.88–0.93). Specificity was estimated at 0.73, 95%CI = (0.61; 0.82). Higher heterogeneity is visually evident for specificity, while sensitivities are more homogeneous. Figure 3 displays the Summary ROC (SROC) curve that demonstrates such variability within a small range of values for sensitivity estimates and within a wider range for specificity estimates. No correlation was found between sensitivities and specificities (ρ = 0.22, IC95% [–0.33; 0.66]). Diagnostic odds ratio was estimated at: DOR = 27.5, 95%CI (14.7; 48.5).

FIGURE 2

Figure 2. Forest plots for sensitivity and specificity. Studies are sorted alphabetically. *Estimated pooled sensitivity and specificity with their 95% confidence intervals from bivariate methods using a random effects model. As no statistical method is currently available for heterogeneity assessment in diagnostic metanalysis with bivariate methods, heterogeneity was assessed qualitatively and considered low for sensitivity and high for specificity. Although high, heterogeneity for specificity estimates was explained by differences between studies and thus considered not serious (see quality of evidence in the results section). Sensitivity analysis excluding studies with concerns regarding flawed reference standard did not change displayed estimates.

FIGURE 3

Figure 3. Summary ROC curve. Estimation of diagnostic accuracy of chest CT for detection of COVID-19: high sensitivity with narrow range of variability and modest specificity (inverse of false positive rate) with a wider range of variability; attributable to variable local COVID-19 incidence and differences in reference standards applied.

Likelihood Ratios and Practical Interpretation of Results

The calculated likelihood ratios derived from pooled estimates of sensitivity and specificity were: LR+ = 3.44 (sensitivity/[1-specificity]) and LR– = 0.13 ([1-sensitivity]/specificity). A practical interpretation of this results can be made using the estimated change in pre-test probability according to calculated LRs provided by McGee (50). The presence of a compatible or highly suspicious chest CT (“positive”) increases pre-test probability by 20–25%, while an unlikely/incompatible (negative) chest CT reduces the pre-test probability of COVID-19 by ∼45%.

Quality of the Evidence

Evidence assessment to estimate diagnostic accuracy of chest CT for COVID-19 was undertaken separately for sensitivity and specificity, as recommended. Both assessments considered 15 studies comprising 4824 patients. Evidence for sensitivity estimates was graded as “Moderate” due to indirectness of evidence, and for specificity estimates it was graded as “Low” due to imprecision (evidenced by a wide 95% confidence interval) and indirectness. The summary of findings (SoF) table (Table 4) displays the judgments made for each aspect of quality assessment and their corresponding explanations. Effect is presented for pre-test probabilities of 1 and 10%.

TABLE 4

Table 4. Summary of findings.

Additional Findings

Three studies specifically addressed using CT as a method of screening for COVID-19 in an emergency surgery setting. In one study of 28 patients with initial negative RT-PCR results that upon a second test became positive, the mean interval time between negative to positive results was 6.2 days, with some up to 15 days (51). In two cases PCR was negative two times before being positive, with positive chest CT findings identifying lesions on the first day. In countries like India where there is an ongoing surge of COVID-19 patients, shortages in test kit supplies have strained health systems. CT testing can compensate for this, especially in situations where the patient needs emergency surgery. In these cases, the urgency to get the patient to the operating room is incompatible with the time needed to receive test results from PCR. This is important as one study found that there was a higher number of positive PCR in the trauma population than in the general population (52). On the contrary, in low-resource areas in which CT scans are not readily available at many hospitals transferring patients to hospitals with CT availability may be a risk to patients and healthcare workers. Another study of 207 patients admitted for acute surgical emergencies found a negative predictive value of 82.4%, concluding that CT of the thorax has the potential to play an important role in helping surgeons in their decision making (53). However, the authors note that over-reliance on CT with its high false positive rate can lead to overtreatment, overuse of resources and delays in the decision-making process. In the third study with data provided for over 800 patients undergoing both emergency and elective surgical interventions over a 5-day period at all UK hospitals with imaging departments, a high rate of false positives was found, producing a sensitivity of 68.4% for thoracic CT (54). These authors suggest that the diagnostic yield is low and that additional CT examinations expose patients to an unnecessary extra dose of radiation to the patient.

The CO-RADS Scale

The most widespread method for the diagnosis of COVID-19 using CT imaging appears to be the COVID-19 Reporting and Data System (CO-RADS), introduced by the Radiological Society of the Netherlands (NVvR) and largely based on the recommendations of the Radiological Society of North America (11, 12). The scoring system uses a scale from 0 to 5 to grade the level of suspicion of COVID-19 infection based on pulmonary involvement from very unlikely to very likely. In a study of 105 patients, the NVvR found high performance for predicting COVID-19 with an AUC of 0.91 (CI, 0.85–0.97) (12, 55). A high negative predictive value and low negative likelihood ratio was associated with a CO-RADS ≤ 3 while a high positive likelihood ratio and good positive predictive value was associated with a CO-RADS score ≥ 4. It is important to note that in both studies the prevalence of COVID-19 was high. As cases drop it is likely that the false-positive rate and negative predictive value will increase. Of the reviewed studies, in four that did not report a dichotomous index test (+ or –) and thus could not be meta-analyzed, the CO-RADS scale was used (12, 33, 38, 56). Dichotomization of the scale was used by one additional study that did report + or – results for the index test.

Interrater Agreement of Chest Computed Tomography for COVID-19

Since diagnostic efficacy is dependent upon the radiologist’s interpretation of the CT scan, the reproducibility of the categorization of CT reports among multiple observers is an essential component when considering appropriate clinical decision making. In a study of 241 COVID-19 suspected patients, eight observers categorized each CT into one of four categories (evocative, compatible for COVID-19 pneumonia, not evocative, and normal) (35). Agreement across the 4 categories was good between all readers (κ value 0.61 95% CI 0.60–0.63) and moderate to good between pairs of readers (0.54–0.75). Among patients considered for hospitalization, CT categorized as evocative is highly predictive of COVID-19, while almost a third of patients with CT categorized as not evocative had a positive RT-PCR. In another study of 34 COVID-19 and 48 non-COVID-19 patients identified by RT-PCR, two radiologists had a good interobserver agreement (κ value 0.69) with 26/34 COVID-19 patients correctly diagnosed at final agreement (48). Since several studies did not examine inter-observer variability of CT findings it is possible that the observed specificity and sensitivity are overestimated.

Discussion

This review identified, appraised, and summarized the available evidence regarding the diagnostic accuracy of chest CT scan for the diagnosis of COVID-19 in order to inform clinical decisions and recommendations regarding its application in an emergency trauma setting. No studies that assessed chest CT diagnostic accuracy in this specific setting were found. Nonetheless, consideration of evidence from the ED setting during the pandemic was planned in advance given that we judged the diagnosis accuracy largely extrapolatable, as is indicated in our published protocol (8). Some case series reporting the utility of CT in the pre-operative screening on COVID-19 were summarized narratively but were not a source for diagnostic accuracy of the test. We report that chest CT is a highly sensitive tool to detect COVID-19 in suspected patients and would be expected to have a similar sensitivity when applied to a trauma patient, but has lower specificity.

Great variation was present between studies due to differences in design, index test definition and reference standards. The nature of the disease and differences in settings likely affected specificity estimates in the included studies. The local incidence of COVID-19 cases explains variability of specificity estimates among studies. When incidence is low, abnormal chest CT findings suggestive of COVID-19 might be more often caused by other etiologies such as other respiratory viruses. Conversely, in a context of high COVID-19 incidence, abnormal findings due to non-COVID-19 pneumonias constitute a smaller proportion of the studied patient. This leads to a lower false positives rate being recorded for the test, and hence, higher calculated specificity. This relationship is to be considered when applying and interpreting results of chest CT for screening or diagnosing COVID-19.

The value of chest CT appears to be that of an additional screening tool that can easily detect PCR false negatives, which are reportedly frequent. It is a sensitive tool to diagnose COVID-19, and specificity can vary as discussed. Carrier/asymptomatic status, which is believed to also represent contagion risk, is obviously not expected to be detected with chest CT. Thus, a negative chest CT does not exclude SARS-CoV-2 transmission in the incubation phase. However, given the absence of COVID-19 pneumonia, the likelihood of contagion is significantly reduced. In this sense, CT can have increased value if reading is compatible with COVID-19 when either PCR testing is unavailable or results are delayed. Compatible findings on chest CT should prompt additional protective measures in aerosolizing procedures for medical staff and isolating measures for the patient should be considered. As emergency trauma patients typically undergo localized or full body CT scanning, imaging of the lungs and its interpretation by a radiologist is not expected to increase costs significantly and can be implemented as a screening tool in that setting. One important consideration are patients with trauma or polytrauma involving the chest; as lung contusions and hemo- or pneumo-thorax will affect the readability of chest CT for pneumonia.

Strengths and Limitations

A considerable proportion of the systematic reviews encountered when conducting this synthesis were found to be of low methodological quality and thus with high risk of bias because they included studies without a DTA design where a reference standard and an index test are applied to all patients, studies of only COVID-19 positive cases and/or did not considered the rate of false negatives on single initial RT-PCR. We provide new estimates that overcome these limitations and that should serve as a trustworthy source of information regarding the diagnostic accuracy of chest CT for COVID-19. We hope that the insights into the methodology of assessing diagnostic tests performance and synthesizing this type of evidence will be of value for readers.

Quality of the evidence for diagnostic accuracy of chest CT was moderate for sensitivity estimates and low for specificity estimates, mostly due to indirect evidence and, in the case of specificity, imprecision. Higher quality of evidence requires studies that assess chest CT in the trauma setting. Chest CT is a sensitive test for COVID-19 that can have a role in screening of trauma patients with need for urgent surgical care; it has easy implementation as CT is routinely performed in trauma patients, and could be particularly useful in low-resources settings where supplies are to be used selectively.

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Author Contributions

All authors listed have made a substantial, direct, and intelectual contribution to the work, and approved it for publication.

Funding

This work was supported by the National Institute for Health Research (NIHR) Global Health Research Group on Neurotrauma, which was commissioned by the NIHR using United Kingdom aid from the United Kingdom Government (project 16/137/105). PH is the chief investigator of the RESCUEicp and RESCUE-ASDH randomized trials. DG was supported by the Gates Cambridge Trust (OPP1144). PH was supported by a Research Professorship from the NIHR, the NIHR Cambridge Biomedical Research Centre, a European Union Seventh Framework Programme grant (CENTER-TBI; grant number, 602,150), and the Royal College of Surgeons of England. AK was supported by a Clinical Lectureship from the School of Clinical Medicine, University of Cambridge, and the Royal College of Surgeons of England. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

CT, computed tomography; ED, emergency department; PRISMA, preferred reporting items for systematic reviews and meta-analyses; LOVE, living overview of the evidence; DTA, diagnostic test accuracy; WHO, World Health Organization; PCR, polymerase chain reaction; UK, United Kingdom; CO-RADS, COVID-19 reporting and data system; NVvR, radiological society of the Netherlands.

References

1. McMahon DE, Peters GA, Ivers LC, Freeman EE. Global resource shortages during COVID-19: bad news for low-income countries. PLoS Neglect Trop D. (2020) 14:e0008412. doi: 10.1371/journal.pntd.0008412

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Cohen J, Rodgers YVM. Contributing factors to personal protective equipment shortages during the COVID-19 pandemic. Prev Med. (2020) 141:106263. doi: 10.1016/j.ypmed.2020.106263

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Kaye AD, Okeagu CN, Pham AD, Silva RA, Hurley JJ, Arron BL, et al. Economic impact of COVID-19 pandemic on healthcare facilities and systems: international perspectives. Best Pract Res Clin Anaesthesiol. (2020) 35:293–306. doi: 10.1016/j.bpa.2020.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Butler-Laporte G, Lawandi A, Schiller I, Yao MC, Dendukuri N, McDonald EG, et al. Comparison of saliva and nasopharyngeal swab nucleic acid amplification testing for detection of SARS-CoV-2: a systematic review and meta-analysis. JAMA Intern Med. (2021) 181:353–60. doi: 10.1001/jamainternmed.2020.8876

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Böger B, Fachi MM, Vilhena RO, Cobre AF, Tonin FS, Pontarolo R. Systematic review with meta-analysis of the accuracy of diagnostic tests for COVID-19. Am J Infect Control. (2020) 49:21–9. doi: 10.1016/j.ajic.2020.07.011

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, Zambrano-Achig P, Campo RD, Ciapponi A, et al. False-negative results of initial RT-PCR assays for COVID-19: a systematic review. PLoS One. (2020) 15:e0242958. doi: 10.1371/journal.pone.0242958

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Suchá D, van Hamersvelt RW, van den Hoven AF, de Jong PA, Verkooijen HM. Suboptimal quality and high risk of bias in diagnostic test accuracy studies at chest radiography and CT in the acute setting of the covid-19 pandemic: a systematic review. Radiol. Cardiothorac Imaging. (2020) 2:e200342. doi: 10.1148/ryct.2020200342

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Griswold D, Gempeler A, Rosseau G, Kaseje N, Johnson WD, Kolias A, et al. Chest computed tomography for the diagnosis of COVID-19 in emergency trauma surgery patients who require urgent care during the pandemic: protocol for an umbrella review. Jmir Res Protoc. (2021) 10:e25207. doi: 10.2196/25207

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Roqué M, Martínez Garcúa L, Solà I, Alonso-Coello1 P, Bonfill1 X, Zamora J, et al. Toolkit of methodological resources to conduct systematic reviews [version 3; peer review: 2 approved]. F1000Research (2020) 9:82. doi: 10.12688/f1000research.22032.3

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg Lond Engl. (2010) 8:336–41. doi: 10.1016/j.ijsu.2010.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Simpson S, Kay FU, Abbara S, Bhalla S, Chung JH, Chung M, et al. Radiological society of north america expert consensus document on reporting chest ct findings related to COVID-19: endorsed by the society of thoracic radiology, the american college of radiology, and RSNA. Radiol. Cardiothorac Imaging. (2020) 2:e200152. doi: 10.1148/ryct.2020200152

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Prokop M, van Everdingen W, van Rees Vellinga T, Quarles van Ufford H, Stöger L, Beenen L, et al. CO-RADS: a categorical CT assessment scheme for patients suspected of having COVID-19-Definition and Evaluation. Radiology. (2020) 296:E97–104. doi: 10.1148/radiol.2020201473

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Epistemonikos Foundation. (0000). Living Evidence Repository for COVID-19. Available online at: https://app.iloveevidence.com/loves/5e6fdb9669c00e4ac072701d.

Google Scholar

14. McGrath TA, McInnes MDF, Korevaar DA, Bossuyt PMM. Meta-Analyses of diagnostic accuracy in imaging journals: analysis of pooling techniques and their effect on summary estimates of diagnostic accuracy. Radiology. (2016) 281:78–85. doi: 10.1148/radiol.2016152229

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. (2001) 20:2865–84. doi: 10.1002/sim.942

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Doebler P, Holling H. Meta-Analysis of Diagnostic Accuracy with mada. R Packag 0.5.10. (2015).

Google Scholar

17. Atkins D, Best D, Briss PA, Eccles M, Falck-Ytter Y, Flottorp S, et al. Grading quality of evidence and strength of recommendations. BMJ. (2004) 328:1490. doi: 10.1136/bmj.328.7454.1490

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. (2010) 64:383–94. doi: 10.1016/j.jclinepi.2010.04.026

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Grade handbook. Handbook for Grading the Quality of Evidence and the Strength of Recommendations Using the GRADE Approach. (2013). Available online at: https://gdt.gradepro.org/app/handbook/handbook.html (accessed April 20, 2021).

Google Scholar

20. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. (2005) 58:882–93. doi: 10.1016/j.jclinepi.2005.01.016

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021) Cochrane (2021)

Google Scholar

22. Adams HJA, Kwee TC, Yakar D, Hope MD, Kwee RM. Systematic Review and Meta-Analysis on the Value of Chest CT in the Diagnosis of Coronavirus Disease (COVID-19): Sol Scientiae, Illustra Nos. AJR Am J Roentgenol. (2020) 215:1342-1350. doi: 10.2214/AJR.20.23391

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Mair MD, Hussain M, Siddiqui S, Das S, Baker A, Conboy P, et al. A systematic review and meta-analysis comparing the diagnostic accuracy of initial RT-PCR and CT scan in suspected COVID-19 patients. Br J Radiology. (2021) 94:20201039. doi: 10.1259/bjr.20201039

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Huang EP-C, Sung C-W, Chen C-H, Fan C-Y, Lai P-C, Huang Y-T. Can computed tomography be a primary tool for COVID-19 detection? Evidence appraisal through meta-analysis. Crit Care Lond Engl. (2020) 24:193. doi: 10.1186/s13054-020-02908-4

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Li K, Wu X, Zhong Y, Qin W, Zhang Z. Diagnostic performance of CT and its key signs for COVID-19: a systematic review and meta-analysis. medRxiv [preprint]. (2020): doi: 10.1101/2020.05.24.20111773

CrossRef Full Text | Google Scholar

26. Khatami F, Saatchi M, Zadeh SST, Aghamir ZS, Shabestari AN, Reis LO, et al. A meta-analysis of accuracy and sensitivity of chest CT and RT-PCR in COVID-19 diagnosis. Sci Rep. (2020) 10:22402. doi: 10.1038/s41598-020-80061-2

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kim H, Hong H, Yoon SH. Diagnostic performance of ct and reverse transcriptase-polymerase chain reaction for coronavirus disease 2019: a meta-analysis. Radiology. (2020) 296:E145–55. doi: 10.1148/radiol.2020201343

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Lv M, Wang M, Yang N, Luo X, Li W, Chen X, et al. Chest computed tomography for the diagnosis of patients with coronavirus disease 2019 (COVID-19): a rapid review and meta-analysis. Ann Transl Med. (2020) 8:622. doi: 10.21037/atm-20-3311

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Shao JM, Ayuso SA, Deerenberg EB, Elhage SA, Augenstein VA, Heniford BT. A systematic review of CT chest in COVID-19 diagnosis and its potential application in a surgical setting. Colorectal Dis. (2020) 22:993–1001. doi: 10.1111/codi.15252

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Xu B, Xing Y, Peng J, Zheng Z, Tang W, Sun Y, et al. Chest CT for detecting COVID-19: a systematic review and meta-analysis of diagnostic accuracy. Eur Radiol. (2020) 30:5720–7. doi: 10.1007/s00330-020-06934-2

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Islam N, Ebrahimzadeh S, Salameh J-P, Kazi S, Fabiano N, Treanor L, et al. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Db Syst Rev. (2021) 2021:CD013639.

Google Scholar

32. Aslan S, Bekçi T, Çakır İM, Ekiz M, Yavuz İY, Çahin AM, et al. Diagnostic performance of low-dose chest CT to detect COVID-19: a turkish population study. Diagn Interv Radiol. (2020) 27:181–7. doi: 10.5152/dir.2020.20350

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Bellini D, Panvini N, Rengo M, Vicini S, Lichtner M, Tieghi T, et al. Diagnostic accuracy and interobserver variability of CO-RADS in patients with suspected coronavirus disease-2019: a multireader validation study. Eur Radiol. (2020) 31:1932–40. doi: 10.1007/s00330-020-07273-y

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Caruso D, Zerunian M, Polici M, Pucciarelli F, Polidori T, Rucci C, et al. Chest CT Features of COVID-19 in Rome, Italy. Radiology. (2020) 296:E79–85. doi: 10.1148/radiol.2020201237

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Debray M-P, Tarabay H, Males L, Chalhoub N, Mahdjoub E, Pavlovsky T, et al. Observer agreement and clinical significance of chest CT reporting in patients suspected of COVID-19. Eur Radiol. (2020) 31:1081–9. doi: 10.1007/s00330-020-07126-8

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Gezer NS, Ergan B, Barış MM, Appak Ö, Sayıner AA, Balcı P, et al. COVID-19 S: a new proposal for diagnosis and structured reporting of COVID-19 on computed tomography imaging. Diagn Interv Radiol Ankara Turk. (2020) 26:315–22. doi: 10.5152/dir.2020.20351

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Gietema HA, Zelis N, Nobel JM, Lambriks LJG, van Alphen LB, Oude Lashof AML, et al. CT in relation to RT-PCR in diagnosing COVID-19 in The Netherlands: a prospective study. PLoS One. (2020) 15:e0235844. doi: 10.1371/journal.pone.0235844

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Fujioka T, Takahashi M, Mori M, Tsuchiya J, Yamaga E, Horii T, et al. Evaluation of the usefulness of CO-RADS for chest CT in patients suspected of having COVID-19. Diagnostics. (2020) 10:608. doi: 10.3390/diagnostics10090608

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Herpe G, Lederlin M, Naudin M, Ohana M, Chaumoitre K, Gregory J, et al. Efficacy of chest CT for COVID-19 pneumonia diagnosis in France. Radiology. (2020) 298:E81–7. doi: 10.1148/radiol.2020202568

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Korevaar DA, Kootte RS, Smits LP, Aardweg JG, Bonta PI, Schinkel J, et al. Added value of chest computed tomography in suspected COVID-19: an analysis of 239 patients. Eur. Respir J. (2020) 56:2001377. doi: 10.1183/13993003.01377-2020

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Krdzalic J, de Jaegere TMH, Kwee RM. Diagnostic performance of chest CT in screening patients with suspected COVID-19 infection in a Western population. Br J Radiol. (2020) 93:20200643. doi: 10.1259/bjr.20200643

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Patel M, Chowdhury J, Zheng M, Abramian O, Verga S, Zhao H, et al. High Resolution CHEST CT(HRCT) evaluation in patients hospitalized with COVID-19 Infection. Medrxiv [preprint]. (2020): doi: 10.1101/2020.05.26.20114082

CrossRef Full Text | Google Scholar

43. Song S, Wu F, Liu Y, Jiang H, Xiong F, Guo X, et al. Correlation between chest CT findings and clinical features of 211 COVID-19 suspected patients in Wuhan, China. Open Forum Infect Dis. (2020) 7:ofaa171. doi: 10.1093/ofid/ofaa171

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Steuwe A, Rademacher C, Valentin B, Koehler M-H, Appel E, Keitel V, et al. Dose optimized chest CT for diagnosis of COVID-19 – Evaluation of the image quality and diagnostic impact. J Radiol Prot. (2020) 40:877–91. doi: 10.1088/1361-6498/aba16a

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Wen Z, Chi Y, Zhang L, Liu H, Du K, Li Z, et al. Coronavirus disease 2019: Initial Detection on Chest CT in a Retrospective Multicenter Study of 103 Chinese Patients. Radiol Cardiothorac Imaging. (2020) 2:e200092. doi: 10.1148/ryct.2020200092

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Xie C, Jiang L, Huang G, Pu H, Gong B, Lin H, et al. Comparison of different samples for 2019 novel coronavirus detection by nucleic acid amplification tests. Int J Infect Dis. (2020) 93:264–7. doi: 10.1016/j.ijid.2020.02.050

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Zhu W, Xie K, Lu H, Xu L, Zhou S, Fang S. Initial clinical features of suspected coronavirus disease 2019 in two emergency departments outside of Hubei, China. J Med Virol. (2020) 92:1525–32. doi: 10.1002/jmv.25763

PubMed Abstract | CrossRef Full Text | Google Scholar

48. He J-L, Luo L, Luo Z-D, Lyu J-X, Ng M-Y, Shen X-P, et al. Diagnostic performance between CT and initial real-time RT-PCR for clinically suspected 2019 coronavirus disease (COVID-19) patients outside Wuhan. China. Resp Med. (2020) 168:105980. doi: 10.1016/j.rmed.2020.105980

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Schulze-Hagen M, Hübel C, Meier-Schroers M, Yüksel C, Sander A, Sähn M, et al. Low-Dose Chest CT for the Diagnosis of COVID-19. Deutsches Aerzteblatt Online. (2020) 117:389–95. doi: 10.3238/arztebl.2020.0389

PubMed Abstract | CrossRef Full Text | Google Scholar

50. McGee S. Simplifying likelihood ratios. J Gen Intern Med. (2002) 17:647–50. doi: 10.1046/j.1525-1497.2002.10750.x

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Hernigou J, Cornil F, Poignard A, Bouchaibi SE, Mani J, Naouri JF, et al. Thoracic computerised tomography scans in one hundred eighteen orthopaedic patients during the COVID-19 pandemic: identification of chest lesions; added values; help in managing patients; burden on the computerised tomography scan department. Int Orthop. (2020) 44:1571–80. doi: 10.1007/s00264-020-04651-5

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Hernigou J, Morel X, Callewier A, Bath O, Hernigou P. Staying home during “COVID-19” decreased fractures, but trauma did not quarantine in one hundred and twelve adults and twenty eight children and the “tsunami of recommendations” could not lockdown twelve elective operations. Int Orthop. (2020) 44:1473–80. doi: 10.1007/s00264-020-04619-5

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Majeed T, Ali RS, Solomon J, Mesri M, Sharma S, Shamim S, et al. The Role of the Computed Tomography (CT) Thorax in the Diagnosis of COVID-19 for patients presenting with acute surgical emergencies. a single institute experience. Indian J Surg. (2020) 82:1005–10. doi: 10.1007/s12262-020-02626-9

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Callaway M, Harden S, Ramsden W, Beavon M, Drinkwater K, Vanburen T, et al. A national UK audit for diagnostic accuracy of preoperative CT chest in emergency and elective surgery during COVID-19 pandemic. Clin Radiol. (2020) 75:705–8. doi: 10.1016/j.crad.2020.06.010

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Lieveld AWE, Azijli K, Teunissen BP, van Haaften RM, Kootte RS, van den Berk IAH, et al. Chest CT in COVID-19 at the ED: validation of the COVID-19 reporting and data system (CO-RADS) and CT severity score. Chest. (2021) 159:1126–35. doi: 10.1016/j.chest.2020.11.026

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Korevaar DA, Kootte RS, Smits LP, van den Aardweg JG, Bonta PI, Schinkel J, et al. Added value of chest CT in Suspected COVID-19: an analysis of 239 patients. Eur Respir J. (2020) 56:2001377. doi: 10.1183/13993003.01377-2020

PubMed Abstract | CrossRef Full Text | Google Scholar