Comprehensive evaluation of plasma microbial cell-free DNA sequencing for predicting bloodstream and local infections in clinical practice: a multicenter retrospective study

Background: Metagenomic next-generation sequencing (mNGS) of plasma cell-free DNA (cfDNA) shows promising application for complicated infections that cannot be resolved by conventional microbiological tests (CMTs). The criteria for cfDNA sequencing are currently in need of agreement and standardization.

Methods: We performed a retrospective cohort observation of 653 patients who underwent plasma cfDNA mNGS, including 431 with suspected bloodstream infections (BSI) and 222 with other suspected systemic infections. Plasma mNGS and CMTs were performed simultaneously in clinical practice. The diagnostic efficacy of plasma mNGS and CMTs in the diagnosis of blood-borne and other systemic infections was evaluated using receiver operating characteristic (ROC) curves. The sensitivity and specificity of the two methods were analyzed based on the final clinical outcome as the gold standard.

Results: The mNGS test showed an overall positive rate of 72.3% (472/653) for detecting microorganisms in plasma cfDNA, with a range of 2 to 6 different microorganisms detected in 171 patient specimens. Patients with positive mNGS results were more immunocompromised and had a higher incidence of severe disease (P<0·05). The sensitivity of mNGS was higher for BSI (93·5%) and other systemic infections (83·6%) compared to CMTs (37·7% and 14·3%, respectively). The mNGS detected DNA from a total of 735 microorganisms, with the number of microbial DNA reads ranging from 3 to 57,969, and a higher number of reads being associated with clinical infections (P<0·05). Of the 472 patients with positive mNGS results, clinical management was positively affected in 203 (43%) cases. Negative mNGS results led to a modified clinical management regimen in 92 patients (14.1%). The study also developed a bacterial and fungal library for plasma mNGS and obtained comparisons of turnaround times and detailed processing procedures for rare pathogens.

Conclusion: Our study evaluates the clinical use and analytic approaches of mNGS in predicting bloodstream and local infections in clinical practice. Our results suggest that mNGS has higher positive predictive values (PPVs) for BSI and systemic infections compared to CMTs, and can positively affect clinical management in a significant number of patients. The standardized whole-process management procedure for plasma mNGS developed in this study will ensure improved pre-screening probabilities and yield clinically valuable data.

Introduction

Bloodstream infections (BSI) are infectious diseases that may endanger the lives of critically ill patients (Rhee et al., 2019) and traditional blood culture methods are the gold standard for diagnosing BSI (Rhodes et al., 2017; Miller et al., 2018). However, blood culture methods can only identify the pathogen in less than 20% of sepsis cases (Miller et al., 2018); most pathogens are not detected in more than half of patients in routine clinical testing (Blauwkamp et al., 2019). In recent years, nucleic acid molecular diagnostic techniques have been increasingly used to directly test blood cultures as a means to improve diagnostic accuracy and accelerate pathogen identification (Marschal et al., 2017). Molecular assays need to predict the presence of possible infectious pathogens in advance and are designed to detect specific targets, which increases the detection bias. These techniques have noticeable deficiencies in clinical practice, especially for rare unknown pathogens, where identification is confirmed with positive blood culture. Metagenomic next-generation sequencing (mNGS) is an advanced technology applied to pathogen diagnostic practice in recent years. It can detect pathogens’ genomes in clinical samples without relying on culture enrichment and amplification (Greninger et al., 2017; Chen et al., 2020). Additionally, mNGS allows detection and identification of a virtually unlimited range of organisms in a single test without prior selection of target pathogens (Gu et al., 2019). In clinical practice, the detection of plasma cell-free DNA mNGS is not commonly used to examine BSI (Carbo et al., 2022). A study by Blauwkamp et al. (2019) showed that the sensitivity of mNGS for sepsis diagnosis is much higher than that of all other methods.

Because blood flows through various tissues and organs throughout the body, even localized infections outside the blood system may have enough pathogenic DNA fragments to enter the bloodstream to be detected by mNGS (Farnaes et al., 2019). A previous study showed that fragments of genomic DNA from pathogens infecting other parts of the body can be detected in purified plasma cell-free DNA (cfDNA) (Song et al., 2021), suggesting the possibility of noninvasive detection of various infections by sequencing plasma microbial cfDNA. Plasma mNGS has many advantages over conventional microbiological tests (CMTs), the most important being able to detect microbial cfDNA fragments in the plasma of patients without BSI. These fragments may be related to commensal microorganisms irrelevant to sepsis or incidentally discovered. However, it’s not possible to distinguish whether the detected cfDNA fragments originate from the responsible infectious pathogen or from transient bacteremia caused by the colonizing bacteria. Clinicians and laboratories alike are facing the challenge of determining the optimal utility of mNGS technology. This challenge arises from the difficulty in establishing the clinical application value and interpreting the results from mNGS. To address this challenge, large-scale clinical investigations are needed to provide insights into the clinical value of mNGS and guide the establishment of guidelines for complete experimental setup. This study retrospectively analyzed plasma mNGS data from 653 subjects, discuss the actual diagnostic effect and clinical application value of the cfDNA method in clinical BSI and other systemic infections, and we propose corresponding diagnosis and treatment suggestions.

Methods

Setting and study design

This was a multicenter, retrospective study. We retrospectively studied the medical records of 668 patients who underwent plasma mNGS sequencing in nine medical institutions, including Liaocheng People’s Hospital, between August 2020 and May 2022. The same instrument, library preparation kit, and data analysis pipeline were used at all 9 institutions for plasma cfDNA mNGS. Patients were excluded from the subject group if they died within 24 hours (n = 12), had insufficient clinical data (n = 2), or did not obtain informed consent (n = 1). A total of 653 patients were finally included in the study (Figure 1). Plasma mNGS testing and CMTs blood cultures were performed simultaneously (Plasma mNGS and CMT procedure section).

Figure 1

Figure 1 Overview of patient enrollment workflow.

Plasma mNGS and CMT procedure

The K2EDTA anticoagulant tubes were used for blood specimen collection. All patients underwent plasma mNGS detection, blood culture, and other CMTs methods. Plasma mNGS was performed using the shotgun method on a BGISEQ-50 platform (BGI Genomics, Wuhan, China) with the PMseq^® high-throughput detection kit of pathogens (MGI Tech Co., Ltd, Wuhan, China). Prior to its use, the method was validated based on BGI’s general verification process, including running mock specimens to determine limits of detection and specificity. Negative controls were included during library construction and sequencing to control for contamination. Sequence data was based on the BGI self-built microbial genome reference database (see Supplementary File 1). Bacterial detection methods generally included microscopic examination, culture, and acid-fast staining for Mycobacterium tuberculosis, among others. Fungal infections were evaluated using 1,3-β-D glucan assay and galactomannan assay, while the Rose Bengal test (RBT) was used for suspected human brucellosis. Common viruses were detected by fluorescence quantitative PCR (qPCR), including Human cytomegalovirus (CMV), Epstein–Barr virus (EBV), Herpes simplex virus (HSV) type 1 and 2, Torque teno virus (TTV), Hepatitis B virus (HBV), JC polyomavirus (JCPyV), BK polyomavirus, and Varicella-zoster virus (VZV). Additional primers were designed and validated by Sanger sequencing in cases where mNGS identified pathogens that were not detected by the above CMT methods.

Pathogen and clinical data analysis

Table 1 provides a general reference guide for clinicians considering the use of plasma metagenomic next-generation sequencing (mNGS) in the diagnosis of sepsis and other systemic infections. The table presents potential clinical indications for plasma mNGS, as well as pre-analytical, analytical, and post-analytical considerations that clinicians may find helpful in deciding when to use plasma mNGS testing and how to interpret the results. These recommendations are based on the current literature and expert opinion, and are not specific to the study. To provide additional clarity regarding the criteria used in this study to define “True positive,” “false positive,” and other parameters used for result adjudication, we referred to Table 2. This table outlines the theoretical routes for clinical interpretation of plasma mNGS-based detection of sepsis and other systemic infections. The research subjects were divided into two groups based on admission diagnosis: (1) suspected bloodstream infection (BSI) (431 cases); and (2) suspected other systemic infection (222 cases; Figure 2A). We separately evaluated the diagnostic efficacy of mNGS and conventional microbiological tests (CMTs) for each group. mNGS detection was considered true positive when the sequencing reads of a pathogenic microorganism detected in plasma were consistent with the clinical diagnosis, including results of microbiological culture and other laboratory tests, as well as the patient’s clinical symptoms and medical history. We used a composite reference method to determine the clinical impact, which included the interpretation of clinical history and all microbiological data, including mNGS findings. To determine the presence of microbial infection, we used comprehensive clinical criteria, including those presented in Supplementary File 2. We calculated clinical impact by microorganism and patient, with each microorganism and plasma specimen interpreted as one object, respectively. The calculations considered the diagnostic efficacy of both mNGS and CMTs, as well as the presence or absence of microbial infection, as determined by our comprehensive clinical criteria.

Table 1

Table 1 Analytical considerations for plasma mNGS in diagnosis of sepsis and other system infections.

Table 2

Table 2 Theoretical routes for clinical interpretation from plasma mNGS-based detection of sepsis and other system infections.

Figure 2

Figure 2 Identification of pathogens by conventional methods and mNGS. (A) Group analysis workflow for the samples included in the study. All subjects were divided into two groups based on admitting diagnosis: bloodstream infection group and other systemic infection group. (B, C) Positivity rate comparison and concordance analysis between metagenomic next-generation sequencing (mNGS) and CMTs for blood system infection and other system infection samples. The number of positive samples (y-axis) for pairwise mNGS and CMTs testing is plotted against the infectious and Non-infectious (x-axis). The Pearson chi-squared (χ2) test were used for the comparison of categorical data frequencies. P < 0.05 indicates a significant difference. (D–G) ROC curves and 2 × 2 contingency tables showing the respective diagnostic performance of mNGS and CMTs for blood system infection and other system infection groups. NPV, negative predictive value; PPV, positive predictive value. (H, I) Comparison of the diagnostic efficacy of mNGS and CMTs methods in blood system infection and other system infection groups using ROC curves. P < 0.05 indicates a significant difference.

Background interference and turnaround time of mNGS

Because mNGS is a very sensitive detection method, contamination may be introduced through the environment, reagents, and processing procedures at any time point during sample collection, nucleic acid extraction, library preparation, and sequencing. To avoid the influence of contamination on the results, we analyzed the main background bacteria and their distribution frequency in mNGS detection. In this study, a total of 70 rounds of mNGS experiments were conducted, with each round including a negative control. In each mNGS experiment, the three background microorganisms (bacteria and fungi were counted once each) with the highest frequency and abundance were counted in both the negative control and negative sample. Subsequently, the dominant background flora were analyzed from 10 rounds of experiments conducted each month and their frequencies were visualized as heat maps. In this study, the turnaround times (TATs) for tests including polymerase chain reaction (PCR), metagenomic next-generation sequencing (mNGS), bacterial culture and identification, and fungal culture and identification were collected and analyzed in the Laboratory Information System (LIS) database. The calculation of TATs began with the receipt of the specimen and ended with the next process or the issuance of the test report.

Results

Clinical characteristics of study patients

The study involved 8·3% (54/653) children and 68·8% (449/653) immunocompromised patients. Regardless of clinical relevance, the overall positive detection rate of mNGS was 72·3% (472/653). Compared to patients with a negative mNGS test, patients with a positive mNGS test were more immunocompromised and had a higher rate of severe disease (P < 0·05, Table 3). Among the mNGS-positive patients, hematological disease, fever of unknown origin (FUO), and chronic obstructive pulmonary disease (COPD) accounted for the top three identified diseases (Table 3).

Table 3

Table 3 Clinical characteristics of study patients (n=653).

Comparison between mNGS and CMTs

The TAT analysis showed that mNGS took less time than CMTs, including culture methods. However, mNGS took longer than methods such as PCR and antibody detection (Figure 3). In both the BSI and other systemic infection groups (Figure 2A), the true positive rate of mNGS diagnosis was greater than that in the CMTs (Figures 2B, C). The diagnostic sensitivity of mNGS was also greater than that of CMTs; however, its diagnostic specificity was less than that of CMTs (Figures 2D–G). We analyzed the diagnostic performance plasma mNGS and CMTs in these two groups. The results showed that mNGS had a receiver operating characteristic (ROC) curve of 0.796 for suspected BSIs, which was significantly higher than that of CMTs (P < 0·01, Figures 2D, H). Both mNGS and CMTs had low diagnostic performance in detecting other systemic infections. The area under curve (AUC) value of mNGS was 0.546 (CMTs = 0.510), with no statistically significant difference between them (Figure 2I).

Figure 3

Figure 3 Timeline for mNGS and CMTs. Culture-based pathogen identification can take days to weeks. In comparison, mNGS testing using the BGI sequencing platforms has an overall turnaround time of 24 h.

Pathogen species and infection categories

We detected a total of 735 suspected causative pathogens in the plasma of 653 patients using mNGS, of which 171 plasma samples showed multiple organisms. Among all 735 microorganisms, we detected 380 bacteria (51·7%), followed by viruses (32·0%). The gram-negative bacilli (GNB) were the most common pathogens and directly associated with approximately 50% of sepsis cases. Community-acquired pneumonia, hospital-acquired pneumonia, accounted for the largest proportion of infections among other systemic infections (Table 4). Among all unconfirmed infection sequences, viruses accounted for the highest proportion of sequences (69·8%, 164/235), which may be related to the latent of EBV or HSV, although their association with diseases remains unclear. Gram-positive bacteria caused an increase in an unconfirmed infection due to clinical uncertainty in coagulase-negative staphylococci. The detailed distribution of organisms detected in the plasma by mNGS is provided in Supplementary File 3. Sequence reads were in the ranges of 3–57969, 3–4454, 3–41266, and 3–914 for bacteria, fungi, viruses, and other rare pathogens, respectively. In addition, we observed a higher number of DNA sequences in clinically relevant cases (Table 5). During the study, a total of 26 special pathogens were identified with reads ranging from 3 to 2290 (Table 6). Some special pathogens can be confirmed by PCR or culture. However, due to their rarity, certain pathogens cannot be confirmed by other methods, and infection can only be diagnosed by comprehensive clinical interpretation.

Table 4

Table 4 Infection categories pointed to by positive microorganisms in plasma detected by mNGS assay.

Table 5

Table 5 Comparison of distributions of sequencing reads for clinically relevant and irrelevant organisms (lines indicate median values) .

Table 6

Table 6 Processing details detected in plasma mNGS for specific pathogens.

Clinical impact and relevance of mNGS

Of all 472 mNGS-positive patients, clinical management of 203 (43·0%) patients was positively affected by mNGS (e.g., new pathogens identified and antibiotic regimen changed) (Table 7). A total of 269 (57·0%) patients did not change their treatment regimen (e.g., pathogen detection of unknown significance, mNGS results were slower than CMT results, or previous antibiotic regimen covered the pathogen) (Table 7). Furthermore, 92 patients received a modified clinical management regimen, including de-escalation of antibiotics and avoidance of invasive tests such as tissue biopsy, based on negative mNGS test results suggesting a low likelihood of pathogen infection. Of the 653 patients who underwent plasma mNGS testing, 295 (45·2%) benefited from the technology. Among the 735 microorganisms detected by mNGS, 489 (66·8%) were not detected by CMTs. Of these, 232 (47·4%) microorganisms led to a change in the clinical management of patients that ultimately benefited the patients.

Table 7

Table 7 Clinical impact of mNGS in plasma by patient (n=653) and by microorganism (n=735).

Background microorganisms

In this study, bacteria mainly caused background interference and reads ranged between 10 and 117. The top three bacteria were Cutibacterium acnes, Micrococcus luteus, and Brevundimonas vesicularis (Figure 4).

Figure 4

Figure 4 Heat map illustrating the common background Microorganisms in mNGS. (A) Background bacteria. (B) Background fungi.

Discussion

Currently, mNGS is not recommended for routine use in mild/moderate infections due to its high cost (Chiu and Miller, 2019). However, it is necessary to carry out adequate development in complex and critically ill patients (Simner et al., 2018). The external clinical manifestations of FUO and other various diseases (Goggin et al., 2020; Yan et al., 2021), especially for different immunocompromised conditions (Carbo et al., 2022), are often detected by plasma mNGS. In our investigation, the utilization of plasma mNGS was found to be effective in identifying instances of bloodstream infections, particularly those caused by uncommon pathogens, thus highlighting the utility of this technology. In addition, our study provided reasonable inclusion criteria that can effectively guide clinicians in selecting tests or assays. The findings also confirmed that immunocompromised and ICU patients were more likely to obtain positive mNGS results. Similar studies have shown that plasma mNGS of ICU samples are effective in diagnosing bacterial BSI (Yan et al., 2021). Additionally, we note that mNGS has advantages over CMTs, especially for fungi and mycobacteria, which further expands the clinical indications of this technology. Hogan et al (Hogan et al., 2021) reported that plasma mNGS provided results faster for three fungal diagnoses, leading to earlier optimization of antifungal therapy. Because our mNGS protocol requires ~ 24 hours of TAT, patients with longer hospital stays may benefit from mNGS, potentially impacting their clinical management plan. Furthermore, we believe that when the clinical treatment response is inconsistent with CMTs, it is often a misjudgment of the infected pathogen, and the timely implementation of mNGS will improve diagnostic efficiency.

Another potential application of plasma mNGS is the identification of other non-blood systemic infections, which is a feasible and new attempt. Li et al (Li et al., 2022) reported that among 109 patients with an abdominal infection, 61 positive cases were detected and clinically confirmed by plasma mNGS. In other reports, plasma mNGS has been helpful in the diagnosis of patients with endocarditis (Kondo et al., 2019; Hogan et al., 2021). In addition, pneumonitis and invasive mycosis have also progressed experimentally by using plasma mNGS (Hong et al., 2018; Farnaes et al., 2019). However, the significance of guiding clinical infection management is still controversial due to the small number of cases using this technology.

A novel and comprehensive study on the application of plasma mNGS method for the detection of systemic infections was conducted. This study differs and surpasses some of the previous studies published in this field in several aspects. Blauwkamp et al. reported a blood cfDNA sequencing-based diagnostic method (Blauwkamp et al., 2019), but their pathogen database was limited to bacteria and fungi, while a larger and more comprehensive database encompassing more types and numbers of microorganisms, such as viruses, parasites, mycoplasma, chlamydia, etc., was used in this study. Their method was only tested on simulated samples, while the method in this study was validated on clinical samples, and compared with conventional methods, demonstrating high consistency and accuracy.

In the realm of infectious disease diagnostics, metagenomic next-generation sequencing (mNGS) stands as a beacon of innovation, offering a broad-spectrum pathogen detection capability that transcends the limitations of traditional culture-based methods (Loman and Pallen, 2015). The principal advantage of mNGS lies in its unparalleled sensitivity and specificity, enabling the identification of a diverse array of pathogens, including bacteria, viruses, fungi, and parasites, from a single sample (Chiu and Miller, 2019). This holistic approach is particularly beneficial in cases of polymicrobial infections, where multiple pathogens coexist, often eluding conventional diagnostic techniques. Moreover, mNGS is adept at detecting novel or rare pathogens, thereby expanding the diagnostic horizon significantly. However, this technology is not without its limitations. The interpretation of mNGS data can be challenging, given the complexity of distinguishing pathogenic sequences from background microbial DNA, particularly in cases of low-level or transient bacteremia (Meyer et al., 2022). Additionally, the high cost and technical expertise required for mNGS limit its widespread adoption in routine clinical practice. Furthermore, while mNGS offers rapid turnaround times compared to traditional cultures, the need for specialized equipment and bioinformatics support can introduce delays (Hasman et al., 2014). Thus, while mNGS heralds a new era in infectious disease diagnostics, its integration into clinical practice necessitates a balanced consideration of its strengths and limitations, underscoring the need for continued research and development in this dynamic field (Rhoads et al., 2012; Lecuit and Eloit, 2015).

In this study, our sizeable clinical database diagnosed 34 cases of fungal pneumonia and one case of tuberculous pneumonia by plasma mNGS, which provides another option for examination in patients who cannot tolerate bronchoalveolar lavage. In addition, we found that the nucleic acid fragments of urinary pathogens by nucleic acid fragments in plasma, such as Neisseria gonorrhoeae. Nevertheless, as demonstrated in this study, the diagnostic accuracy of plasma mNGS for infections in other body systems is relatively low. Tissue or body fluids are still recommended for local infections due to the high local concentration of pathogens. Therefore, deriving local infections from plasma mNGS can only be used as an alternative since not all pathogenic nucleic acids can spread to the blood.

The mNGS technique exhibits high sensitivity compared to CMTs, which was confirmed in our study. A similar study examined the value of mNGS in predicting BSI in childhood leukemia patients and demonstrated that the assay had a predictive sensitivity and specificity of 75% and 82%, respectively (Goggin et al., 2020). We believe this is also related to patient inclusion criteria, as mNGS procedures are often performed only when there is a high suspicion of infection in the current situation. The decreased specificity of mNGS compared to conventional molecular techniques can result in incorrect diagnoses and treatments in clinical settings. To address this, mNGS results are validated using other techniques, and improvements in mNGS technologies and bioinformatics algorithms are ongoing to improve specificity. However, it is still important to note that “true detection” does not necessarily mean “true infection,” Pathogens (such as EBV) confirmed by PCR in this study cannot be used to guide clinical management directly due to their unclear infection characteristics. A report on the use of mNGS in 386 patients with community-acquired sepsis of unknown etiology in Vietnam demonstrated that a human-infectious virus was detected in 32/384 serum samples. Whether there is a direct clinical association with sepsis remains unclear (Anh et al., 2019). In addition, opportunistic pathogens with unconfirmed infection, such as coagulase-negative staphylococci, have not been shown to have an effective clinical impact. In addition, the presence of multiple mixed pathogens in our study plasma specimens and the relationship between them is very complex. Zanella et al (Zanella et al., 2021) performed mNGS analysis on 25 adult allogeneic hematopoietic stem cell transplantation recipients and detected sequences from ≥ 3 different viruses in 16/25 patients. The relationship between pathogens of different species remains to be determined. The read abundance of different pathogens can be used to interpret disease species load (Carbo et al., 2022), but only for homogeneous pathogens. Different pathogens display distinctive cell wall characteristics and this impacts the efficiency of nucleic acid extraction; thus, a high abundance of organisms does not necessarily equal clinical causation. Therefore, when clinicians report multiple pathogens from a single mNGS specimen, clinical decision-making needs to comprehensively interpret the report based on microorganisms’ nucleic acid acquisition efficiency and pathogenicity.

Views on the impact of mNGS on the final clinical protocol vary and are related to differences between studies, including patient populations, study designs, and definitions used to determine clinical impact. Data from the Children’s Hospital of Chicago showed that mNGS was considered to provide clinically relevant information in 56% of submitted samples (Rossoff et al., 2019). Wilke et al (Wilke et al., 2021) conducted a retrospective chart review of 142 cell-free mNGS test results from a tertiary pediatric hospital and confirmed that 32·4% of the results were directly applicable to clinical management change. In contrast, Niles et al (Niles et al., 2020) showed that in most cases, little or no additional value was gained when mNGS was ordered concurrently with routine testing. A good clinical correlation rate was obtained in our clinical practice due to the following aspects.

First, many patients underwent mNGS testing even when CMT results were negative, which was found to be more likely to result in a positive diagnosis. Second, some subjects in this study with local infections were not able to provide local specimens due to limitations and culture testing was not performed. Reliance on plasma mNGS was more effective in detecting new infections. Third, the value of mNGS was demonstrated in the discovery of pathogenic bacteria in follow-up tracing, such as the case of urinary tract infection caused by Ureaplasma parvum that we observed. Fourth, the importance of mNGS in clinical practice is reflected in the timely diagnosis of patients, such as eliminating the possibility of co-infections in rheumatism patients. These factors validate the clinical impact of mNGS, primarily through the effective application of antibiotics. Because the cause of infection in patients was not identified in the early stage, and increasing antibiotics were used, the misuse of antibiotics was effectively avoided after mNGS detected the pathogen. Similar studies have confirmed that the positive effects of mNGS are mainly driven by the de-escalation of antimicrobial therapy (Farnaes et al., 2019). In addition, we found that change in clinical management rates were lower than the actual mNGS proven infection rates, which is likely related to the fact that many patients have empirically selected antimicrobials that already cover current pathogens.

This study encountered limitations in the application of the mNGS technique. Firstly, variations in mNGS platforms could result in disparities in the number of reads and levels of abundance for the same specimen, leading to disparate interpretations and incongruent final outcomes. Even when utilizing the same platform, modifications to the amount of sequencing data per sample can impact the reads obtained from the specimen (Diao et al., 2022). Our findings were in line with prior studies, which indicated that high read counts are indicative of infection and that increasing the cutoff value might hypothetically elevate the clinical compliance rate. However, it should be noted that such an increase could also reduce analytical sensitivity and potentially eliminate some true positive results (Carbo et al., 2022). To ensure adequate sensitivity of detection, a standard control sample was sequenced to a depth of 20 million reads on our platform. The presence of infection was also established with just three reads, demonstrating that there is not a single universally accepted threshold. On the other hand, it is possible that a pathogen may go undetected if its sequencing reads are very low (less than three), which reflects its low concentration in plasma. In cases where other sources of disseminated infection are reflected in plasma mNGS, this low yield of sequencing data becomes less effective. Such low pathogen concentrations are also unlikely to be confirmed by culturing methods. While augmenting the sequencing depth per sample might increase the detection rate, it is also crucial to consider the increased cost and potential increase in background noise.

Second, the organisms detected by mNGS experiments may originate from preanalytical processes or from clinically unrelated organisms (Lee et al., 2020). Although these conditions are generally considered false positives, it is sometimes difficult to distinguish between true infection or colonization by normal flora. Particularly in immunocompromised patients, the human microbiota is more likely to translocate into the bloodstream leading to infection (Horiba et al., 2021). Studies have demonstrated that contaminating nucleic acids can be quantified to identify false-positive fragments using the mNGS process (Zinter et al., 2019; Jing et al., 2021), but different sequencing platforms and reagents result in different contaminating sequences (Diao et al., 2022). In this study, we established our own background microbial library, which was beneficial to reduce the interference of common contaminating sequences. It should be noted that different laboratories need to establish their own background bacterial libraries and update them regularly, especially when there are changes in instruments or reagents.

Third, despite the excellent true-negative parameters provided by the mNGS test (Lee et al., 2020), there were still some false-negatives in our experiments, especially for plasma viruses that were lower compared with the real-time PCR method. In addition, this study did not involve the detection of RNA, which may miss some RNA viruses in patients with unexplained fever. Software processing of sequencing data can also cause false negative results. Because the exported sequencing data are often sorted according to reads or abundance, high-interference sequences can cause the true pathogenic bacteria to rank lower and not be automatically exported. Data processing errors can also occur, such as database alignment errors. Positive results were deleted by mistake due to the same organism deriving from the negative control. At present, it is still difficult to formulate specific screening rules for each sample. When the data, automatically exported by the software, do not match the clinical data, the logout list of each sequencing run should be carefully reviewed. Microbiologists can identify sequences that may be clinically significant. Additionally, mNGS testing in our subjects could be performed before and after antimicrobial treatment. In theory, dead organisms have little effect on molecular detection methods, but the reduction in the overall DNA amount after the number of pathogens is reduced may also have an impact on the results. Fourth, although some clinical effects indicate positive manifestations, it does not rule out that in certain circumstances, physicians have no choice but to “believe” the mNGS results, which may not necessarily be accurate, and make changes in clinical antimicrobial therapy accordingly. Finally, we believe that there is still a large oversight in assessing antimicrobial susceptibility or resistance using current methods and warrants future investigations.

Conclusion

The mNGS test is technically complex and no guidelines are available to help with the interpretation of the results. Before the technology is further developed and running costs reduced, it is necessary to emphasize the establishment of strict access standards, control of various parameters of the detection process (including wet and dry experiments), and professional review and multidisciplinary discussion during interpretation. This will ensure improved pre-screening probabilities and yield clinically meaningful, value-added data.

Data availability statement

The data presented in the study are deposited in the CNGB Sequence Archive (CNSA) of China National GeneBank DataBase (CNGBdb), accession number CNP0003204 (https://db.cngb.org/search/project/CNP0003204/).

Ethics statement

The studies involving humans were approved by Medical Ethics Committee of Liaocheng People’s Hospital. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

FP: Conceptualization, Writing – original draft, Writing – review & editing. WX: Data curation, Writing – original draft, Writing – review & editing. HZ: Data curation, Investigation, Writing – original draft. SC: Data curation, Writing – original draft. YT: Data curation, Writing – original draft. JF: Methodology, Writing – original draft. ZY: Data curation, Writing – review & editing. PS: Formal Analysis, Writing – original draft. QX: Writing – review & editing. QZ: Writing – review & editing. XJ: Writing – review & editing. CW: Data curation, Formal Analysis, Investigation, Methodology, Software, Writing – original draft, Writing – review & editing.

Funding

This work was supported by the Medical Science and Technology Development Foundation of Department of Health of Shandong Province, China (2019WS113).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcimb.2023.1256099/full#supplementary-material

References

Anh, N. T., Hong, N. T. T., Nhu, L. N. T., Thanh, T. T., Lau, C. Y., Limmathurotsakul, D., et al. (2019). Viruses in Vietnamese patients presenting with community-acquired sepsis of unknown cause. J. Clin. Microbiol. 57 (9), e00386-19. doi: 10.1128/JCM.00386-19

PubMed Abstract | CrossRef Full Text | Google Scholar

Blauwkamp, T. A., Thair, S., Rosen, M. J., Blair, L., Lindner, M. S., Vilfan, I. D., et al. (2019). Analytical and clinical validation of a microbial cell-free DNA sequencing test for infectious disease. Nat. Microbiol. 4 (4), 663–674. doi: 10.1038/s41564-018-0349-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Carbo, E. C., Russcher, A., Kraakman, M. E. M., de Brouwer, C. S., Sidorov, I. A., Feltkamp, M. C. W., et al. (2022). Longitudinal monitoring of DNA viral loads in transplant patients using quantitative metagenomic next-generation sequencing. Pathogens 11 (2), 236. doi: 10.3390/pathogens11020236

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, L., Liu, W., Zhang, Q., Xu, K., Ye, G., Wu, W., et al. (2020). RNA based mNGS approach identifies a novel human coronavirus from two individual pneumonia cases in 2019 Wuhan outbreak. Emerg. Microbes Infect. 9 (1), 313–319. doi: 10.1080/22221751.2020.1725399

PubMed Abstract | CrossRef Full Text | Google Scholar

Chiu, C. Y., Miller, S. A. (2019). Clinical metagenomics. Nat. Rev. Genet. 20 (6), 341–355. doi: 10.1038/s41576-019-0113-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Diao, Z. L., Zhang, R., Li, J. M. (2022). [Analysis of the methods and quality assurance of metagenomic next-generation sequencing to detect the microbial cfDNA from blood samples in China]. Zhonghua Yi Xue Za Zhi 102 (15), 1114–1118. doi: 10.3760/cma.j.cn112137-20220104-00017

PubMed Abstract | CrossRef Full Text | Google Scholar

Farnaes, L., Wilke, J., Ryan Loker, K., Bradley, J. S., Cannavino, C. R., Hong, D. K., et al. (2019). Community-acquired pneumonia in children: cell-free plasma sequencing for diagnosis and management. Diagn. Microbiol. Infect. Dis. 94 (2), 188–191. doi: 10.1016/j.diagmicrobio.2018.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Goggin, K. P., Gonzalez-Pena, V., Inaba, Y., Allison, K. J., Hong, D. K., Ahmed, A. A., et al. (2020). Evaluation of plasma microbial cell-free DNA sequencing to predict bloodstream infection in pediatric patients with relapsed or refractory cancer. JAMA Oncol. 6 (4), 552–556. doi: 10.1001/jamaoncol.2019.4120

PubMed Abstract | CrossRef Full Text | Google Scholar

Greninger, A. L., Zerr, D. M., Qin, X., Adler, A. L., Sampoleo, R., Kuypers, J. M., et al. (2017). Rapid metagenomic next-generation sequencing during an investigation of hospital-acquired human parainfluenza virus 3 infections. J. Clin. Microbiol. 55 (1), 177–182. doi: 10.1128/JCM.01881-16

PubMed Abstract | CrossRef Full Text | Google Scholar

Gu, W., Miller, S., Chiu, C. Y. (2019). Clinical metagenomic next-generation sequencing for pathogen detection. Annu. Rev. Pathol. 14, 319–338. doi: 10.1146/annurev-pathmechdis-012418-012751

PubMed Abstract | CrossRef Full Text | Google Scholar

Hasman, H., Saputra, D., Sicheritz-Ponten, T., Lund, O., Svendsen, C. A., Frimodt-Møller, N., et al. (2014). Rapid whole-genome sequencing for detection and characterization of microorganisms directly from clinical samples. J. Clin. Microbiol. 52 (1), 139–146. doi: 10.1128/JCM.02452-13

PubMed Abstract | CrossRef Full Text | Google Scholar

Hogan, C. A., Yang, S., Garner, O. B., Green, D. A., Gomez, C. A., Dien Bard, J., et al. (2021). Clinical impact of metagenomic next-generation sequencing of plasma cell-free DNA for the diagnosis of infectious diseases: A multicenter retrospective cohort study. Clin. Infect. Dis. 72 (2), 239–245. doi: 10.1093/cid/ciaa035

PubMed Abstract | CrossRef Full Text | Google Scholar

Hong, D. K., Blauwkamp, T. A., Kertesz, M., Bercovici, S., Truong, C., Banaei, N. (2018). Liquid biopsy for infectious diseases: sequencing of cell-free plasma to detect pathogen DNA in patients with invasive fungal disease. Diagn. Microbiol. Infect. Dis. 92 (3), 210–213. doi: 10.1016/j.diagmicrobio.2018.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Horiba, K., Torii, Y., Okumura, T., Takeuchi, S., Suzuki, T., Kawada, J. I., et al. (2021). Next-generation sequencing to detect pathogens in pediatric febrile neutropenia: A single-center retrospective study of 112 cases. Open Forum Infect. Dis. 8 (11), ofab223. doi: 10.1093/ofid/ofab223

PubMed Abstract | CrossRef Full Text | Google Scholar

Jing, C., Chen, H., Liang, Y., Zhong, Y., Wang, Q., Li, L., et al. (2021). Clinical evaluation of an improved metagenomic next-generation sequencing test for the diagnosis of bloodstream infections. Clin. Chem. 67 (8), 1133–1143. doi: 10.1093/clinchem/hvab061

PubMed Abstract | CrossRef Full Text | Google Scholar

Kondo, M., Dalai, S. C., Venkatasubrahmanyam, S., Eisenberg, N., Robinson, B. D., Westblade, L. F., et al. (2019). Diagnosis and genotyping of Coxiella burnetii endocarditis in a patient with prosthetic pulmonary valve replacement using next-generation sequencing of plasma microbial cell-free DNA. Open Forum Infect. Dis. 6 (6), ofz242. doi: 10.1093/ofid/ofz242

PubMed Abstract | CrossRef Full Text | Google Scholar

Lecuit, M., Eloit, M. (2015). The potential of whole genome NGS for infectious disease diagnosis. Expert Rev. Mol. Diagn 15 (12), 1517–1519. doi: 10.1586/14737159.2015.1111140

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, R. A., Al Dhaheri, F., Pollock, N. R., Sharma, T. S. (2020). Assessment of the clinical utility of plasma metagenomic next-generation sequencing in a pediatric hospital population. J. Clin. Microbiol. 58 (7), e00419-20. doi: 10.1128/JCM.00419-20

PubMed Abstract | CrossRef Full Text | Google Scholar

Li, D., Gai, W., Zhang, J., Cheng, W., Cui, N., Wang, H. (2022). Metagenomic next-generation sequencing for the microbiological diagnosis of abdominal sepsis patients. Front. Microbiol. 13, 816631. doi: 10.3389/fmicb.2022.816631

PubMed Abstract | CrossRef Full Text | Google Scholar

Loman, N. J., Pallen, M. J. (2015). Twenty years of bacterial genome sequencing. Nat. Rev. Microbiol. 13 (12), 787–794. doi: 10.1038/nrmicro3565

PubMed Abstract | CrossRef Full Text | Google Scholar

Marschal, M., Bachmaier, J., Autenrieth, I., Oberhettinger, P., Willmann, M., Peter, S. (2017). Evaluation of the accelerate Pheno system for fast identification and antimicrobial susceptibility testing from positive blood cultures in bloodstream infections caused by gram-negative pathogens. J. Clin. Microbiol. 55 (7), 2116–2126. doi: 10.1128/JCM.00181-17

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, F., Fritz, A., Deng, Z. L., Koslicki, D., Lesker, T. R., Gurevich, A., et al. (2022). Critical Assessment of Metagenome Interpretation: the second round of challenges. Nat. Methods 19 (4), 429–440. doi: 10.1038/s41592-022-01431-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Miller, J. M., Binnicker, M. J., Campbell, S., Carroll, K. C., Chapin, K. C., Gilligan, P. H., et al. (2018). A guide to utilization of the microbiology laboratory for diagnosis of infectious diseases: 2018 update by the infectious diseases society of America and the American society for microbiology. Clin. Infect. Dis. 67 (6), e1–e94. doi: 10.1093/cid/ciy381

PubMed Abstract | CrossRef Full Text | Google Scholar

Niles, D. T., Wijetunge, D. S. S., Palazzi, D. L., Singh, I. R., Revell, P. A. (2020). Plasma metagenomic next-generation sequencing assay for identifying pathogens: a retrospective review of test utilization in a large children's hospital. J. Clin. Microbiol. 58 (11), e00794-20. doi: 10.1128/JCM.00794-20

PubMed Abstract | CrossRef Full Text | Google Scholar

Rhee, C., Jones, T. M., Hamad, Y., Pande, A., Varon, J., O'Brien, C., et al. (2019). Prevalence, underlying causes, and preventability of sepsis-associated mortality in US acute care hospitals. JAMA Netw. Open 2 (2), e187571. doi: 10.1001/jamanetworkopen.2018.7571

PubMed Abstract | CrossRef Full Text | Google Scholar

Rhoads, D. D., Wolcott, R. D., Sun, Y., Dowd, S. E. (2012). Comparison of culture and molecular identification of bacteria in chronic wounds. Int. J. Mol. Sci. 13 (3), 2535–2550. doi: 10.3390/ijms13032535

PubMed Abstract | CrossRef Full Text | Google Scholar

Rhodes, A., Evans, L. E., Alhazzani, W., Levy, M. M., Antonelli, M., Ferrer, R., et al. (2017). Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016. Intensive Care Med. 43 (3), 304–377. doi: 10.1007/s00134-017-4683-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Rossoff, J., Chaudhury, S., Soneji, M., Patel, S. J., Kwon, S., Armstrong, A., et al. (2019). Noninvasive diagnosis of infection using plasma next-generation sequencing: A single-center experience. Open Forum Infect. Dis. 6 (8), ofz327. doi: 10.1093/ofid/ofz327

PubMed Abstract | CrossRef Full Text | Google Scholar

Simner, P. J., Miller, S., Carroll, K. C. (2018). Understanding the promises and hurdles of metagenomic next-generation sequencing as a diagnostic tool for infectious diseases. Clin. Infect. Dis. 66 (5), 778–788. doi: 10.1093/cid/cix881

PubMed Abstract | CrossRef Full Text | Google Scholar

Song, P., Chen, S., Tan, X., Gao, Y., Fu, J., You, Z., et al. (2021). Metagenomic analysis identifying a rare leishmania infection in an adult with AIDS. Front. Cell Infect. Microbiol. 11, 764142. doi: 10.3389/fcimb.2021.764142

PubMed Abstract | CrossRef Full Text | Google Scholar

Wilke, J., Ramchandar, N., Cannavino, C., Pong, A., Tremoulet, A., Padua, L. T., et al. (2021). Clinical application of cell-free next-generation sequencing for infectious diseases at a tertiary children's hospital. BMC Infect. Dis. 21 (1), 552. doi: 10.1186/s12879-021-06292-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Yan, G., Liu, J., Chen, W., Chen, Y., Cheng, Y., Tao, J., et al. (2021). Metagenomic next-generation sequencing of bloodstream microbial cell-free nucleic acid in children with suspected sepsis in pediatric intensive care unit. Front. Cell Infect. Microbiol. 11, 665226. doi: 10.3389/fcimb.2021.665226

PubMed Abstract | CrossRef Full Text | Google Scholar

Zanella, M. C., Cordey, S., Laubscher, F., Docquier, M., Vieille, G., Van Delden, C., et al. (2021). Unmasking viral sequences by metagenomic next-generation sequencing in adult human blood samples during steroid-refractory/dependent graft-versus-host disease. Microbiome 9 (1), 28. doi: 10.1186/s40168-020-00953-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Zinter, M. S., Mayday, M. Y., Ryckman, K. K., Jelliffe-Pawlowski, L. L., DeRisi, J. L. (2019). Towards precision quantification of contamination in metagenomic sequencing experiments. Microbiome 7 (1), 62. doi: 10.1186/s40168-019-0678-6

PubMed Abstract | CrossRef Full Text | Google Scholar