Skip to main content

ORIGINAL RESEARCH article

Front. Immunol., 16 February 2023
Sec. Viral Immunology

Performance of an interferon-γ release assay-based test for cell-mediated immunity to SARS-CoV-2

Luís Fonseca Brito,Luís Fonseca Brito1,2Silvia TdterSilvia Tödter1Julian KottlauJulian Kottlau1Kathrin CermannKathrin Cermann3Anthea SpierAnthea Spier3Elina Petersen,Elina Petersen4,5Ines Schfer,Ines Schäfer4,5Raphael Twerenbold,,Raphael Twerenbold4,5,6Martin AepfelbacherMartin Aepfelbacher3Marc LütgehetmannMarc Lütgehetmann3Felix R. Stahl,*Felix R. Stahl1,2*
  • 1Institute of Clinical Chemistry and Laboratory Medicine, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
  • 2Department of Virus-Host-Interaction, Leibniz Institute of Virology, Hamburg, Germany
  • 3Institute of Microbiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
  • 4Department of Cardiology, University Heart and Vascular Center, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
  • 5Population Health Research Department, University Heart and Vascular Center, Hamburg, Germany
  • 6German Center for Cardiovascular Research (DZHK) Partner Site Hamburg–Kiel–Lübeck, Hamburg, Germany

In search for immunological correlates of protection against acute coronavirus disease 2019 (COVID-19) there is a need for high through-put assays for cell-mediated immunity (CMI) to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection. We established an interferon-γ release assay -based test for detection of CMI against SARS-CoV-2 spike (S) or nucleocapsid (NC) peptides. Blood samples obtained from 549 healthy or convalescent individuals were measured for interferon-γ (IFN-γ) production after peptide stimulation using a certified chemiluminescence immunoassay. Test performance was calculated applying cutoff values with the highest Youden indices in receiver-operating-characteristics curve analysis and compared to a commercially available serologic test. Potential confounders and clinical correlates were assessed for all test systems. 522 samples obtained from 378 convalescent in median 298 days after PCR-confirmed SARS-CoV-2 infection and 144 healthy control individuals were included in the final analysis. CMI testing had a sensitivity and specificity of up to 89% and 74% for S peptides and 89% and 91% for NC peptides, respectively. High white blood cell counts correlated negatively with IFN-γ responses but there was no CMI decay in samples obtained up to one year after recovery. Severe clinical symptoms at time of acute infection were associated with higher measures of adaptive immunity and reported hair loss at time of examination. This laboratory-developed test for CMI to SARS-CoV-2 NC peptides exhibits excellent test performance, is suitable for high through-put routine diagnostics, and should be evaluated for clinical outcome prediction in prospective pathogen re-exposure.

Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection induces a strong adaptive immune response including production of virus-specific antibodies and T cell immunity (1). Accordingly, besides direct detection of acute SARS-CoV-2 infection via polymerase chain reaction (PCR) or pathogen antigens via rapid lateral flow tests, indirect measures such as detection of endogenous antibodies to SARS-CoV-2 are useful to assess prior pathogen exposure or vaccination (2, 3). Detection of antigen-specific antibodies to epitopes of the SARS-CoV-2 spike (S) or nucleocapsid (NC) proteins are an established diagnostic approach to provide evidence of immunity after infection or vaccination and there are numerous commercially manufactured diagnostic tests available. In fact, effectiveness of antibodies to neutralize virus replication in vitro have been used to assess vaccine responses and predict protection from coronavirus disease 2019 (COVID-19) (47). Although the presence of anti-SARS-CoV-2 antibodies correlates with protection from severe COVID-19, there is strong evidence that antibodies do not mediate sterile immunity and prevent from infection or re-infection with this pathogen (8). The underlying mechanism of protection is likely mediated by additional components of the immune system and in this respect, T cell immunity has been implied as a critical determinant to protect from COVID-19-related hospitalization and death (911). High-throughput cellular assays to measure adaptive SARS-CoV-2 immunity are not commonly requested by physicians and in general less available than serologic testing. Potential reasons are higher costs, extended sample processing times, a more delicate pre-analytic process, and the necessity for higher efforts in developing cell-mediated immunity (CMI) assays. Despite these disadvantages there is a general need to assess cell-mediated adaptive immunity to specific pathogens such as Mycobacterium tuberculosis (12), for individuals who cannot acquire a sufficient humoral immune response due to underlying conditions, or to evaluate responses to chronic infectious diseases (13, 14). Several aspects such as potential correlates of protection from re-infection, broad range recognition of antigens, and detection of long-term immunity argue that SARS-CoV-2 CMI-based tests may be critical for upcoming pandemic-related clinical problems (15).

Here, we used an interferon-γ release assay (IGRA) that uses peptide pools of the S and NC proteins together with a certified in vitro diagnostic analyzer to detect SARS-CoV-2 CMI in a large clinical cohort. We assessed CMI assay performance and potential confounders, compared these results with a commercially available test for anti-NC antibodies, and correlated these measures of adaptive immunity with clinical symptoms.

Materials and methods

Clinical cohort

This study was performed similarly to the post-SARS-CoV-2 cohort previously reported in (16). Individuals living in the metropolitan area of Hamburg, Germany, who had a confirmed positive polymerase chain reaction (PCR) test for SARS-CoV-2 at least 2 months prior to study enrollment were included as convalescent individuals. In parallel, participants of the Hamburg City Health Study (HCHS) (17) with no history of previous SARS-Cov-2 infection and were enrolled during the same time period served as healthy controls. Participants without history of PCR-confirmed SARS-CoV-2 infection but a positive antibody test to SARS-CoV-2 nucleocapsid protein were not excluded from the healthy control group. Participants were 43 - 80 years old at the time of recruitment. All individuals, underwent the identical clinical examination program of the HCHS, which included a blood withdrawal and a questionnaire (only convalescent group) between November 16th 2020 and April 28th 2021. Categorization of disease severity at time of acute infection and potential long COVID-19 symptoms was based on information provided by participants in this questionnaire. Potential re-exposition after a primary SARS-CoV-2 infection was not addressed in the convalescent group. At the time of study enrolment, all participants provided written informed consent. The local ethics committee (State of Hamburg Chamber of Medical Practitioners, PV5131) had no objections against recruitment of post-SARS-CoV-2 individuals as an extension of the HCHS, and the study was conducted in compliance with the Declaration of Helsinki.

CMI

Peptide pools of overlapping 15-mer sequences covering the complete SARS-CoV-2 nucleocapsid (PepTivator® SARS-CoV-2 Prot_N, Miltenyi Biotec), immunodominant sequence domains of SARS-CoV-2 spike (PepTivator® SARS-CoV-2 Prot_S, Miltenyi Biotec) protein, HCMV IE1 (PepTivator® CMV IE-1, Miltenyi Biotec) and pp65 (PepTivator® CMV pp65, Miltenyi Biotec) proteins were used for stimulation of peripheral blood T cells. Within the first 24h after blood collection, 800µL heparinized whole blood was mixed with either phosphate buffered saline (negative control), or 50 ng/mL phorbol-12-myristate-13-acetate (PMA) together with 500 ng/mL ionomycin (positive control), or 1.25 µg/mL SARS-CoV-2 peptides, or 0.625 µg/mL HCMV IE1 together with 0.625 µg/mL HCMV pp65 peptides for 24h at 37°C. IFN-γ concentration was measured in supernatants with a chemiluminescence immunoassay (QuantiFERON®) on a LIAISON® XL analyzer (DiaSorin), as recommended by the manufacturer.

Antibodies, clinical chemistry, hematology

Clinical chemistry and hematology parameters were determined with Atellica® Solution and ADVIA 2120i (Siemens Healthineers), respectively. Anti-NC antibodies (Elecsys anti-SARS-CoV-2) were measured with cobas® e411 (Roche) (18), as recommended by the manufacturer. The majority of recruited convalescent individuals was tested at the University Medical Center Hamburg-Eppendorf for acute SARS-CoV-2 infection with PCR as described in (19) and only these were included for correlation of Ct values to adaptive immunity.

Statistical analysis

Statistical analysis was performed with Prism (GraphPad) using the tests as indicated. Dimensionality reduction was performed with R (version 4.2.1) using the Rtsne package (version 0.16).

Results

We acquired CMI to SARS-CoV-2 S and NC peptides from 549 individuals recruited 62 to 386 days post infection from November 16th 2020 to April 28th 2021 and analyzed IFN-ɣ concentration in blood samples stimulated with either a negative (buffer) or positive (mitogen stimulation) control (Figures 1A, B). We found a higher median baseline IFN-ɣ concentration in negative control samples of convalescent individuals than in the healthy control group (0.0509 and 0.0437 IU/ml, p < 0.0001) whereas responses to mitogen stimulation was generally strong and at the upper measurement range (Figure 1B). In order to identify individual samples that respond insufficiently to mitogen stimulation we calculated the ratio between positive and negative control values and excluded samples with a value below 1.0 as “non-responder”. 522 samples obtained from 378 convalescent patients and 144 controls were included in the final analysis for test quality characteristics of the laboratory developed CMI tests (Figure 1A). In convalescent individuals, blood was collected in median 298 [Q1 229, Q3 319] days after the first PCR-confirmed SARS-CoV-2 infection (Figure 1C). The median age was 55 [Q1 51, Q3 60] years in convalescents and 53 [Q1 49, Q3 59] years in controls. The healthy control group contained less female individuals than the convalescent group (37.5% vs. 48.4%). Sex had no effect on baseline IFN-ɣ concentration, positive controls, and subsequent CMI testing as described below (data not shown and Table 1). 27 samples were excluded from further analysis, 25 due to prior vaccination, and two samples did not pass positive control criteria (Figure 1A).

FIGURE 1
www.frontiersin.org

Figure 1 Study cohort characteristics and Interferon-ɣ release upon stimulation with control agents (A) Individuals were recruited and tested for cell-mediated immunity (CMI) to SARS-CoV-2 as indicated. 27 samples were excluded, 25 due to prior vaccination and two because they did not pass positive control requirements. 522 samples were included in CMI test performance analysis with cohort characteristics as described in (C), data is provided by median and interquartile range. (B) Interferon (IFN)-ɣ concentration of all 549 samples in response to negative (neg. ctrl., buffer) or positive control (pos. ctrl., mitogen) stimulation. Dotted line in pos. ctrl. ratio indicates cut-off of 1 which need to be passed for inclusion in final CMI analysis. Differences between healthy control (hc) and convalescent (conv) group were assessed by Mann Whitney test and p-value is provided above each diagram.

TABLE 1
www.frontiersin.org

Table 1 Summary of potential confounders for anti-SARS-CoV-2 CMI and antibody testing (A) Correlation or direct comparison of cohort characteristics and additional laboratory parameters to measurements of SARS-CoV-2 CMI and antibody testing.

IFN-ɣ concentration in blood samples stimulated with either SARS-CoV-2 spike (S) or nucleocapsid (NC) peptide pools were significantly higher in the convalescent than in the healthy control group (Figures 2A, B). We applied Receiver-Operating-Characteristics (ROC) curves to test for diagnostic abilities and found area under curve values of 0.8334 (0.7889 - 0.8779, 95%CI) and 0.9357 (0.9125 - 0.9589, 95%CI) for S and NC stimulated samples, respectively (Figures 2A, B). Accordingly, using cutoff values with the highest Youden indices in ROC analysis we found a sensitivity and specificity of 89% (86 - 92%, 95%CI) and 69% (61 - 77%, 95%CI) for the CMI test using S peptides whereas stimulation with NC peptides had the same sensitivity of 89% (85 - 92%, 95%CI) but a superior specificity of 87% (80 - 92%, 95%CI) (Figure 2C). We incorporated the differences in baseline IFN-ɣ concentration by calculating ratios of values measured in stimulated to respective negative control samples and re-analyzed the dataset (Figures 2D, E). Using this approach exhibited area under curve values of 0.8184 (0.7736 - 0.8633, 95%CI) and 0.9278 (0.9022 - 0.9534, 95%CI) for S and NC CMI testing (Figures 2D, E). We found a decrease of sensitivity to 88% for both, S (84 - 91%, 95%CI) and NC (84 - 91%, 95%CI) CMI testing, whereas specificity increased to 74% (66 - 81%, 95%CI) and 91% (85 - 95%, 95%CI), respectively (Figure 2C). Within this cohort we additionally performed anti-NC antibody testing of 364 convalescent patients and 140 healthy controls and calculated test performance characteristics according to the manufacturer’s cutoff index (COI) (Figure 2F). We found an area under curve value of 0.9616 (0.9375 to 0.9857, 95%CI) with a sensitivity and specificity of 94% (91 - 96%, 95%CI) and 94% (89 - 98%, 95%CI), respectively (Figures 2C, F). Together, detection of CMI against S and NC peptides exhibited very good to excellent diagnostic performance for detection of previous exposure to SARS-CoV-2. CMI testing with peptides of the NC was superior to those of the S protein and incorporation of baseline IFN-ɣ concentration improved diagnostic specificity of both tests. Nevertheless, all test systems analyzed here apparently exhibited false positive and false negative results.

FIGURE 2
www.frontiersin.org

Figure 2 Test performance characteristics of cell-mediated immunity and antibody testing to SARS-CoV-2 Cell-mediated immunity (CMI) measured in healthy control (hc) and convalescent (conv) groups by supernatant interferon (IFN)-ɣ concentration in response to stimulation with peptide pools covering sequences of the SARS-CoV-2 (A) spike (S) or (B) nucleocapsid (NC) proteins. Ratios to respective negative controls are depicted in (D) for S- and (E) for NC- stimulated samples. (C) Test performance characteristics for CMI and antibody tests determined for cutoff values with highest Youden indices or provided by commercial manufacturer (anti-NC). Sensitivity (sens.) and specificity (spec.) are provided in median and interquartile range for each test application. (F) Anti-NC antibody titers provided as cutoff indices (COI). A+B, (D–F) Differences between healthy control (hc) and convalescent (conv) group were assessed by the Mann-Whitney test and p-value is provided above each diagram, dotted lines indicate corresponding cutoff value. Receiver operating characteristic curves and according area under the curve are provided for each test system.

We aimed to analyze correlative features of the various techniques for detection of SARS-CoV-2 exposure. We found several samples of the healthy control group to be false positive in CMI measurements but true negative by anti-NC testing and vice versa (Figure 3A). Some samples obtained from convalescent patients could be detected as positive by anti-NC only but not by CMI testing and a few were detected by CMI testing only but were negative for anti-NC antibodies. Interestingly, several samples of the healthy control group tested positive for both CMI and antibodies. To gain more insight we used a dimensional reduction approach for visualizing the values of anti-NC, CMI S and NC ratios and found the samples to position in two main and several minor clusters (Figure 3B). Next, we superimposed the test results of all diagnostic methods in a binary format (positive or negative) using the previously calculated cutoff values (Figure 3C). Summation of the test results (negative = 0 and positive = 1) allowed visualizing concordance between the diagnostic tests (Figure 3D). We found 77% of the samples to have a sum of 6 or 0, implying a definite classification of samples to be obtained by patients either with or without prior SARS-CoV-2 infection, respectively. This approach allowed detection of several samples that were categorized as positive only by the CMI S test explaining the lower test specificity if compared to CMI NC (Figures 3C, D). Moreover, numerous samples obtained from convalescent patients were measured as positive via anti-NC, albeit with a rather low COI, but were below the CMI S and NC cutoff values (Figures 3B–D). Hypothesizing a confounding factor to be responsible for this observation, we analyzed additional routine laboratory diagnostic parameters for correlation to anti-NC or CMI measurements. We found higher white blood cell (WBC) counts in these false negative than in true positive CMI tests (Figure 3E). In general, there was a negative correlation between WBC counts and CMI IFN-ɣ responses (Figure 3F; Table 1). Accordingly, high WBC counts seemed to be a disruptive factor for CMI testing whereas anti-NC titers were not affected.

FIGURE 3
www.frontiersin.org

Figure 3 Comparative analysis of test applications for detection of SARS-CoV-2 infection (A) Absolute values measured for each CMI test and anti-NC are depicted as indicated. Dotted lines indicate cutoff values as calculated or provided by commercial manufacturer (anti-NC). (B) Data integration approach for dimensionality reduction and illustration of values (normalized) for each measured sample as indicated. (C) Test result as positive (pos.) or negative (neg.) after evaluation using calculated or provided cutoffs for each test system as indicated. (D) Concordance between test results of anti-NC, CMI S and NC ratios as well as absolute IFN-ɣ values, and PCR by summation of each result (negative = 0 and positive = 1) and superimposition in t-SNE plot as calculated in B). The pie chart illustrates the relative distribution of 100% agreement between test results (sum of 0 = 18% or 6 = 59%) as well as discrepancies (1 to 5). (E) Absolute number of white blood cell (WBC) counts measured in individuals at the same time as acquisition of anti-SARS-CoV-2 adaptive immunity testing. Values are superimposed in t-SNE plot and samples with high test concordance (sum of 6) considered as true positive (t.p) are framed in pink whereas samples with discordance between CMI tests versus antibody and PCR test (a fraction of the group with a sum of 2) are considered as false negative (f.n.) framed in green. Absolute WBC counts for these two subgroups are provided below, differences were assessed by Mann Whitney test and p-value is provided. (F) Correlation of WBC counts versus measurements as indicated. Calculated Spearman correlation results are provided for each diagram, lines indicate simple non-linear regression curves.

Next, we further analyzed cohort characteristics and various diagnostic laboratory parameters of the convalescent group for effects on SARS-CoV-2 antibody or CMI testing (Table 1). There was a negative correlation between anti-NC titers and the time since positive PCR testing for SARS-CoV-2 whereas CMI testing was not significantly affected (Figure 4A; Table 1). Next, we split the cohort into early and late time post infection to analyze for potential differences in immunity. We found significantly higher antibody and IFN-ɣ responses in individuals who were infected less the six months before blood analysis than in those who had recovered more than 6 moths (Figure 4B). In contrast, there was no correlation between PCR cycle threshold (Ct) values and measures of adaptive immunity indicating that virus loads at time of acute infection had no effect on the test systems applied in this study (Table 1). Moreover, we found sex, body mass index (BMI), diabetes, dyslipidemia, or smoking to have no robust effect on CMI testing although there was a minor significant positive correlation between CMI S testing and age (Table 1). In contrast, there was a positive correlation between age, BMI, and cystatin C with anti-NC titers. Moreover, samples obtained from patients with diabetes or dyslipidemia exhibited higher anti-NC titers indicating that metabolic aberrations were associated with higher antibody responses. We additionally screened 31 samples for cytomegalovirus (HCMV) seroprevalence and excluded any effect on the diagnostic interpretation of CMI testing. However, we found a significantly higher median baseline IFN-γ concentration in negative control samples of HCMV+ than HCMV- individuals (0.06460 versus 0.04630 [IFN-ɣ IU/ml], p = 0.0268) and a trend to higher median anti-NC antibody titers in HCMV+ individuals (66.71 versus 28.99 [COI], p = 0.0705) (Table 1). Together, humoral immunity to SARS-CoV-2 appeared to be affected by several conditions that had no significant impact on testing for CMI.

FIGURE 4
www.frontiersin.org

Figure 4 Cell-mediated immunity and antibody testing to SARS-CoV-2 in dependence of time between acute infection and measurement (A) Correlation of days since positive PCR testing versus measurements of CMI and antibodies as indicated. Calculated Spearman correlation results are provided for each diagram, blue lines indicate simple non-linear regression curves, dotted lines indicate cut-off values. (B) Comparative analysis of immunity in individuals infected less or more than six months before data acquisition. The Mann-Whitney test and p-value is provided above each diagram, dotted lines indicate corresponding cutoff value.

Finally, we aimed to correlate clinical symptoms present at time of acute infection or blood withdrawal with IFN-ɣ production and antibody titers. There were higher responses in CMI testing with S and NC peptides as well as anti-NC titers for patients who had severe symptoms or were admitted to hospital during the acute infection phase (Figure 5A). Convalescent individuals were asked for potential long COVID-19 symptoms (Figure 5B) at time of visit for CMI testing but there was no difference in IFN-ɣ production in CMI testing between asymptomatic or symptomatic individuals (Figure 5C). In contrast, we found higher anti-NC titers in symptomatic than asymptomatic patients (Figure 5C), due to higher anti-NC titers in those who developed hair loss several weeks after acute infection (Figure 5D). Moreover, patients who exhibited severe symptoms at time of acute SARS-CoV-2 infection had an increased risk of developing hair loss (Figure 5E). Other post COVID-19 symptoms were not associated with the applied measures of adaptive immunity. Together, severe symptoms at time of acute infection were associated with higher IFN-ɣ production, antibody titers, and the risk of hair loss.

FIGURE 5
www.frontiersin.org

Figure 5 Association of acute and chronic clinical course of SARS-CoV-2 infection on measurements of adaptive immunity (A) Clinical course during acute SARS-CoV-2 infection with corresponding IFN-ɣ responses to peptide stimulation or antibody titers. Difference between groups was assessed by Kruskal-Wallis with Dunn’s multiple comparisons test and p-values below 0.05 are provided above each diagram. The number of individuals per group included in the analysis are provided within the diagrams. (B) List of post COVID symptoms retrieved in the questionnaire at time of sample acquisition and (C) test results in dependence of absence (no) or presence (yes) of symptoms. (D) Anti-NC titers of asymptomatic individuals and those who developed hair loss post SARS-CoV-2 infection. (E) Clinical course during acute SARS-CoV-2 infection of individuals with absence (no) or presence (yes) of hair loss. Differences in (C-E) were assessed by Mann Whitney test and p-value is provided.

Discussion

This study evaluated the usage of an IGRA as a diagnostic test for CMI against SARS-CoV-2 in a large cohort of 522 individuals within the first months of the pandemic prior to the main vaccine program in Germany. At the time of data acquisition there was still a low SARS-CoV-2 seroprevalence of approximately 4% with 3.362.316 and 71.465 cases reported until April 28th 2021 in Germany and Hamburg, respectively (20). Accordingly, in this time window healthy controls with low likelihood of previous asymptomatic SARS-CoV-2 infection or vaccination could be recruited. This is critical in order to determine cutoff values for CMI tests as with increasing seroprevalence and presence long-term adaptive cellular immunity against SARS-CoV-2 there will be a lack of adequate samples as negative controls for assessment of new diagnostic applications in the future.

Although several studies have reported on SARS-CoV-2 CMI using IGRA- or ELISpot-based test systems (2123), the data presented here make its contribution since i) a large number of samples were analyzed for CMI and antibodies, ii) there was a long time period between infection to analysis, iii) a comparative use of S or NC peptides was performed, and iv) test results could be correlated to acute and late clinical symptoms. Accordingly, this study provides additional aspects on how to use IGRA-based CMI tests in the future to evaluate adaptive immunity directed to SARS-CoV-2. For example, for distinguishing between individuals inoculated with S-based vaccines or a prior SARS-CoV-2 infection the use of peptides derived from other than the S protein will be useful and NC peptides match these requirements. Although serologic testing with corresponding specificity can similarly provide this information, individuals who cannot acquire a regular humoral immune response could benefit from the availability of a CMI-based diagnostic test. Moreover, individuals with inborn or acquired immune-deficiencies, such as individuals receiving chemotherapy, a B cell depleting medication, or individuals living with human immunodeficiency virus infection may be tested for presence of cellular immunity via this IGRA-based assay. Furthermore, in newborns there is a diagnostic window where both, false negative, due to lack of antibody response, or false positive serologic testing, as a result of pre- or postnatally transmitted maternal immunoglobulin, are possible (24). In contrast, early life T cell immunity can be detected in the postnatal phase because fetal T cells exist from the 2nd trimester. Indeed, as a proof of principle, this has been shown for congenital HCMV infection (25), and thus testing for CMI may be a useful diagnostic approach to detect SARS-CoV-2 infection in newborns (26). Nevertheless, T cell immunity and IFN- γ responses may be lower in the early life phase (27) and different cut-off values may apply for the CMI tests.

We found a higher baseline IFN-γ concentration in convalescent individuals than in the healthy control group even months after acute SARS-CoV-2 infection. Interestingly, HCMV+ individuals also had a higher IFN-γ plasma concentration than HCMV- study participants. HCMV, a β-herpesvirus is an opportunistic pathogen that leads to lifetime persistent infection (28) whereas SARS-CoV-2 is considered to cause only transient infection. Thus, a chronic inflammatory response with high frequency of activated lymphocytes and according elevated blood baseline IFN-γ concentration in HCMV+ individuals is comprehensible. In contrast, residual immune activation in convalescent SARS-CoV-2 patients with elevated blood baseline IFN-γ concentration even months after acute infection is not expected and future studies will need to decipher the causes and consequences of this observation.

CMI NC testing exhibited performance characteristics with almost comparative features as a commercially available anti-NC antibody-based diagnostic test. In contrast to serologic assays which apparently exhibit comparable performance independent from antibody specificity to S or NC (29), the use of S peptides for CMI led to inferior specificity for detection of SARS-CoV-2 infection. Apparently, in a recent study dominant immunogenic HLA-DR SARS-CoV-2 T cell epitopes were mainly found to be derived from the NC protein (30). Thus, we speculate that the observed higher median production of IFN-γ (0.656 versus 0.260 [IU/ml], p < 0.0001) after stimulation with NC or S peptides, respectively was due to more frequent recognition of NC-specific CD4 T cells in peripheral blood. Accordingly, a higher magnitude of IFN-γ responses in ratio to healthy controls led to a better discrimination performance of the CMI NC test. However, we also found several samples to be tested falsely positive only with the CMI S but not with CMI NC or anti-NC. Children are supposed to develop cross-reactive immunity to SARS-CoV-2 after exposure to endemic human coronaviruses (hCoVs) with a deviated and sustaining response to S peptides (31). However, to our knowledge, this has not been shown for adults and we can only speculate that prior infection with hCoVs may have caused these cellular immune responses to S peptides (32, 33). Similarly, individuals who were assigned to the healthy control group but measured as positive by all CMI and anti-NC tests may have experienced a prior asymptomatic SARS-CoV-2 infection. It needs to be mentioned that individuals included in the current study were infected with early variants of SARS-CoV-2 and only a minor fraction may have been exposed to the variant of concern Alpha (B1.1.7), whereas the variants Delta and Omicron were not present in Germany at this time. The emergence of new variants affect the performance of anti-S more than that of anti-NC antibody tests (34), whereas T cell immunity is less affected by these virus mutations and preserved after natural infection and vaccination (11, 35). This argues that CMI-based diagnostic tests may have favorable performance if compared to serologic test for assessment of infection history with different SARS-CoV-2 variants. Nevertheless, at time of data acquisition there was a SARS-CoV-2 seroprevalence below 1% in Germany, there were very few circulating variants, and accordingly the likelihood for silent re-infection in the convalescent group was rather low. Future studies will need to assess the effects of virus re-exposure and vaccination on CMI testing.

We found high WBC counts in peripheral blood to reduce lymphocyte responses to SARS-CoV-2 peptide stimulation leading to several false negative CMI measurements. Although data on differential blood counts was not available in this study, high WBC counts are usually the result of increased numbers of neutrophils and one can speculate that these cells may be causative for the false negative CMI test results. Indeed, although under debate, there is evidence that neutrophils can interfere with cytokine production of T cells (36). However, in the current study we did not assess which and how leukocytes may interfere with the CMI assay. Nevertheless, leukocyte numbers in peripheral blood need to be considered as a confounding factor for borderline test results in future CMI test applications.

There was a positive correlation between anti-NC titers, age, and several conditions found in patients with metabolic syndrome such as high BMI, dyslipidemia, diabetes, and increased cystatin C plasma concentration as an indicator for reduced glomerular filtration rates. Obesity and peak anti-SARS-CoV-2 titers after infection was observed previously (37). In contrast, high age and obesity is associated with lower antibody titers in response to SARS-CoV-2 mRNA vaccines (38). Moreover, antibody titers were observed to not differ between diabetic and healthy individuals in the early phase after SARS-CoV-2 infection (39). Thus, the observed higher anti-NC titers in this study need confirmation by additional study cohorts with focus on long-term immunity and further investigation to understand the underlying mechanism. Interestingly, we found severe symptoms during acute SARS-CoV-2 infection to be associated with loss of hair and individuals with this post COVID-19 symptom exhibited higher anti-NC titers. This observation is in line with recent reports of transient hair loss after severe COVID-19 (40, 41), a known phenomenon observed after severe infectious disease (42). In this cohort, there was no association between CMI and post COVID-19 symptoms. However, a severe clinical course in the acute infection phase led to subsequent higher T cell IFN-γ secretion and future studies need to assess the relevance of this observation for potential long-term organ alterations and the post-acute COVID-syndrome (43).

A gradual decay in SARS-CoV-2-specific antibody titers in convalescent individuals has been described (44) and we could confirm this observation in the present study. We also found lower antibody and IFN-ɣ levels in samples obtained from individuals who were infected more than half a year before blood withdrawal than those with a more recent infection (Figure 4B) indicating that both, antibody and T cell responses wane over time. This statistical analysis needs to be interpreted with caution as there were 3.3-fold more individuals in the late than in the early response group. Moreover, there was no significant decay in IFN-ɣ production when including all samples without subdividing into the two groups (Figure 4A) and this is in line with observations by recent studies (45). Together, this argues that immunity to SARS-CoV-2 declines over time but T cell immunity likely outlasts plasma antibody titers. This could be a significant advantage of CMI to antibody testing in conditions where long-term adaptive immunity needs to be assessed.

Together, we characterized an in-house developed laboratory test which exhibits excellent test performance, is suitable for high through-put routine diagnostics, and applicable to laboratories with equal instrument equipment. The use of this diagnostic test should be considered in clinical conditions with impaired antibody responses such as innate and acquired immune deficiencies or the early life phase where endogenous cannot be differentiated from maternal immunoglobulin. Moreover, detection of CMI may be useful for future pandemic-related questions such as assessment of long-term adaptive immunity and clinical course after SARS-CoV-2 re-infection.

Data availability statement

The datasets presented in this article are not readily available because requests for HCHS dataset need to be addressed to the corresponding committee. Requests to access the datasets should be directed to RT, r.twerenbold@uke.de.

Ethics statement

The studies involving human participants were reviewed and approved by Hamburg Chamber of Medical Practitioners (PV5131). The patients/participants provided their written informed consent to participate in this study.

Author contributions

LFB and FRS established the CMI test; LFB, ST, JK, KC, AS, MA, and ML conducted CMI, PCR, and antibody data acquisition; FRS and LFB analyzed the data; IS and RT coordinated the study cohort; EP coordinated study population data; FRS conceived the project and wrote the manuscript. All authors reviewed the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Deutsche Forschungsgemeinschaft (STA 1549/2-1 and STA 1549/1-2) and the Stiftung für Pathobiochemie und Molekulare Diagnostik (German Society of Clinical Chemistry and Laboratory Medicine) to FRS. We acknowledge financial support from the Open Access Publication Fund of UKE - Universitätsklinikum Hamburg-Eppendorf- and DFG – German Research Foundation. The underlying Post-COVID project was specifically supported by Senat und Behörde für Wissenschaft, Forschung, Gleichstellung und Bezirke (BWFGB) [Reference E43026-03.HCHS]; Hamburg and the Deutsche Forschungsgemeinschaft [Grant Number TH1106/5-1; AA93/2-1]. The HCHS is generally funded by the euCanSHare grant agreement [Grant Number 825903-euCanSHare H2020]; the Foundation Leducq [Grant Number 16 CVD 03] and the Innovative medicine initiative [Grant Number 116074]. The HCHS is additionally supported by the Deutsche Gesetzliche Unfallversicherung (DGUV); Deutsches Krebsforschungszentrum (DKFZ); Deutsches Zentrum für Herz-Kreislauf-Forschung (DZHK); Deutsche Stiftung für Herzforschung; Seefried Stiftung; Amgen; Bayer; Novartis; Schiller; Siemens; Topcon, and Unilever. The study is further supported by donations from the “Förderverein zur Förderung der HCHS e.V.”, and TePe® (2014). The funders were not involved in the study design, collection, analysis, interpretation of data, the writing of this article, or the decision to submit it for publication.

Conflict of interest

RT reports speaker honoraria/consulting honoraria from Abbott, Amgen, Astra Zeneca, Roche, Siemens and Singulex.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. Sette A, Crotty S. Adaptive immunity to SARS-CoV-2 and COVID-19. Cell (2021) 184(4):861–80. doi: 10.1016/j.cell.2021.01.007

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Kilic T, Weissleder R, Lee H. Molecular and immunological diagnostic tests of COVID-19: Current status and challenges. iScience. (2020) 23(8):101406. doi: 10.1016/j.isci.2020.101406

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Shi J, Han D, Zhang R, Li J, Zhang R. Molecular and serological assays for SARS-CoV-2: Insights from genome and clinical characteristics. Clin Chem (2020) 66(8):1030–46. doi: 10.1093/clinchem/hvaa122

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Feng S, Phillips DJ, White T, Sayal H, Aley PK, Bibi S, et al. Correlates of protection against symptomatic and asymptomatic SARS-CoV-2 infection. Nat Med (2021) 27(11):2032–40. doi: 10.1038/s41591-021-01540-1

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Sahin U, Muik A, Vogler I, Derhovanessian E, Kranz LM, Vormehr M, et al. BNT162b2 vaccine induces neutralizing antibodies and poly-specific T cells in humans. Nature. (2021) 595(7868):572–7. doi: 10.1038/s41586-021-03653-6

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Anderson EJ, Rouphael NG, Widge AT, Jackson LA, Roberts PC, Makhene M, et al. Safety and immunogenicity of SARS-CoV-2 mRNA-1273 vaccine in older adults. N Engl J Med (2020) 383(25):2427–38. doi: 10.1056/NEJMoa2028436

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Khan K, Karim F, Cele S, Reedoy K, San JE, Lustig G, et al. Omicron infection enhances delta antibody immunity in vaccinated persons. Nature. (2022) 607(7918):356–9. doi: 10.1038/s41586-022-04830-x

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Yewdell JW. Individuals cannot rely on COVID-19 herd immunity: Durable immunity to viral disease is limited to viruses with obligate viremic spread. PloS Pathog (2021) 17(4):e1009509. doi: 10.1371/journal.ppat.1009509

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Moss P. The T cell immune response against SARS-CoV-2. Nat Immunol (2022) 23(2):186–93. doi: 10.1038/s41590-021-01122-w

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Cox RJ, Brokstad KA. Not just antibodies: B cells and T cells mediate immunity to COVID-19. Nat Rev Immunol (2020) 20(10):581–2. doi: 10.1038/s41577-020-00436-4

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Liu J, Yu J, McMahan K, Jacob-Dolan C, He X, Giffin V, et al. CD8 T cells contribute to vaccine protection against SARS-CoV-2 in macaques. Sci Immunol (2022) 7:eabq7647. doi: 10.1126/sciimmunol.abq7647

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mazurek GH, Jereb J, Vernon A, LoBue P, Goldberg S, Castro K, et al. Updated guidelines for using interferon gamma release assays to detect mycobacterium tuberculosis infection - united states, 2010. MMWR Recomm Rep (2010) 59(RR-5):1–25.

PubMed Abstract | Google Scholar

13. Hall VG, Humar A, Kumar D. Utility of cytomegalovirus cell-mediated immunity assays in solid organ transplantation. J Clin Microbiol (2022) 60(8):e0171621. doi: 10.1128/jcm.01716-21

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Capretti MG, Marsico C, Chiereghin A, Gabrielli L, Aceti A, Lazzarotto T. Immune monitoring using QuantiFERON(R)-CMV assay in congenital cytomegalovirus infection: Correlation with clinical presentation and CMV DNA load. Clin Infect Dis (2021) 73(3):367–73. doi: 10.1093/cid/ciaa704

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Vardhana S, Baldo L, Morice WG 2nd, Wherry EJ. Understanding T cell responses to COVID-19 is essential for informing public health strategies. Sci Immunol (2022) 7(71):eabo1303. doi: 10.1126/sciimmunol.abo1303

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Petersen EL, Gossling A, Adam G, Aepfelbacher M, Behrendt CA, Cavus E, et al. Multi-organ assessment in mainly non-hospitalized individuals after SARS-CoV-2 infection: The Hamburg city health study COVID programme. Eur Heart J (2022) 43(11):1124–37. doi: 10.1093/eurheartj/ehab914

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Jagodzinski A, Johansen C, Koch-Gromus U, Aarabi G, Adam G, Anders S, et al. Rationale and design of the Hamburg city health study. Eur J Epidemiol (2020) 35(2):169–81. doi: 10.1007/s10654-019-00577-4

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Montalvo Villalba MC, Sosa Glaria E, Rodriguez Lay LLA, Valdes Ramirez O, Vallina Garcia D, Arencibia Garcia A, et al. Performance evaluation of elecsys SARS-CoV-2 antigen immunoassay for diagnostic of COVID-19. J Med Virol (2022) 94(3):1001–8. doi: 10.1002/jmv.27412

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Pfefferle S, Reucher S, Norz D, Lutgehetmann M. Evaluation of a quantitative RT-PCR assay for the detection of the emerging coronavirus SARS-CoV-2 using a high throughput system. Euro Surveill (2020) 25(9). doi: 10.2807/1560-7917.ES.2020.25.9.2000152

PubMed Abstract | CrossRef Full Text | Google Scholar

20. COVID-19-Dashboard. Robert Koch-Institut (2022).

Google Scholar

21. Guo L, Wang G, Wang Y, Zhang Q, Ren L, Gu X, et al. SARS-CoV-2-specific antibody and T-cell responses 1 year after infection in people recovered from COVID-19: a longitudinal cohort study. Lancet Microbe (2022) 3(5):e348–e56. doi: 10.1016/S2666-5247(22)00036-2

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Murugesan K, Jagannathan P, Pham TD, Pandey S, Bonilla HF, Jacobson K, et al. Interferon-gamma release assay for accurate detection of severe acute respiratory syndrome coronavirus 2 T-cell response. Clin Infect Dis (2021) 73(9):e3130–e2. doi: 10.1093/cid/ciaa1537

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Huzly D, Panning M, Smely F, Enders M, Komp J, Steinmann D. Validation and performance evaluation of a novel interferon-γ release assay for the detection of SARS-CoV-2 specific T-cell response. Journal of Clinical Virology (2021). doi: 10.1016/j.jcv.2022.105098

CrossRef Full Text | Google Scholar

24. Jennewein MF, Abu-Raya B, Jiang Y, Alter G, Marchant A. Transfer of maternal immunity and programming of the newborn immune system. Semin Immunopathol (2017) 39(6):605–13. doi: 10.1007/s00281-017-0653-x

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Marchant A, Appay V, van der Sande M, Dulphy N, Liesnard C, Kidd M, et al. Mature CD8(+) T lymphocyte response to viral infection during fetal life. J Clin Invest (2003) 111(11):1747–55. doi: 10.1172/JCI200317470

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Tallarek AC, Urbschat C, Fonseca Brito L, Stanelle-Bertram S, Krasemann S, Frascaroli G, et al. Inefficient placental virus replication and absence of neonatal cell-specific immunity upon sars-CoV-2 infection during pregnancy. Front Immunol (2021) 12:698578. doi: 10.3389/fimmu.2021.698578

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Wang X, Mou W, Qi Z, Chen X, Zhang H, Jiao H, et al. Neonates are armed with deviated immune cell proportion and cytokine reduction but higher T cell proliferation potentiality. Acta Biochim Biophys Sin (2018) 50(9):934–7. doi: 10.1093/abbs/gmy079

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Arvin A, Campadelli-Fiume G, Mocarski E, Moore PS, Roizman B, Whitley R, et al. Human herpesviruses: Biology, therapy, and immunoprophylaxis. (2011) (Cambridge: Cambridge University).

Google Scholar

29. Fox T, Geppert J, Dinnes J, Scandrett K, Bigio J, Sulis G, et al. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev (2022) 11(11):CD013652. doi: 10.1002/14651858.CD013652.pub2

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Nelde A, Bilich T, Heitmann JS, Maringer Y, Salih HR, Roerden M, et al. SARS-CoV-2-derived peptides define heterologous and COVID-19-induced T cell recognition. Nat Immunol (2021) 22(1):74–85. doi: 10.1038/s41590-020-00808-x

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Dowell AC, Butler MS, Jinks E, Tut G, Lancaster T, Sylla P, et al. Children develop robust and sustained cross-reactive spike-specific immune responses to SARS-CoV-2 infection. Nat Immunol (2022) 23(1):40–9. doi: 10.1038/s41590-021-01089-8

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Sekine T, Perez-Potti A, Rivera-Ballesteros O, Stralin K, Gorin JB, Olsson A, et al. Robust T cell immunity in convalescent individuals with asymptomatic or mild COVID-19. Cell (2020) 183(1):158–68.e14. doi: 10.1016/j.cell.2020.08.017

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Loyal L, Braun J, Henze L, Kruse B, Dingeldey M, Reimer U, et al. Cross-reactive CD4(+) T cells enhance SARS-CoV-2 immune responses upon infection and vaccination. Science (2021) 374(6564):eabh1823. doi: 10.1126/science.abh1823

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Springer DN, Perkmann T, Jani CM, Mucher P, Pruger K, Marculescu R, et al. Reduced sensitivity of commercial spike-specific antibody assays after primary infection with the SARS-CoV-2 omicron variant. Microbiol Spectr (2022) 10(5):e0212922. doi: 10.1128/spectrum.02129-22

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Keeton R, Tincho MB, Ngomti A, Baguma R, Benede N, Suzuki A, et al. T Cell responses to SARS-CoV-2 spike cross-recognize omicron. Nature. (2022) 603(7901):488–92. doi: 10.1038/s41586-022-04460-3

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Minns D, Smith KJ, Hardisty G, Rossi AG, Gwyer Findlay E. The outcome of neutrophil-T cell contact differs depending on activation status of both cell types. Front Immunol (2021) 12:633486. doi: 10.3389/fimmu.2021.633486

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Soffer S, Glicksberg BS, Zimlichman E, Efros O, Levin MA, Freeman R, et al. The association between obesity and peak antibody titer response in COVID-19 infection. Obesity (2021) 29(9):1547–53. doi: 10.1002/oby.23208

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Pellini R, Venuti A, Pimpinelli F, Abril E, Blandino G, Campo F, et al. Initial observations on age, gender, BMI and hypertension in antibody responses to SARS-CoV-2 BNT162b2 vaccine. EClinicalMedicine. (2021) 36:100928. doi: 10.1016/j.eclinm.2021.100928

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Lampasona V, Secchi M, Scavini M, Bazzigaluppi E, Brigatti C, Marzinotto I, et al. Antibody response to multiple antigens of SARS-CoV-2 in patients with diabetes: An observational cohort study. Diabetologia. (2020) 63(12):2548–58. doi: 10.1007/s00125-020-05284-4

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Guarnieri G, Bertagna De Marchi L, Marcon A, Panunzi S, Batani V, Caminati M, et al. Relationship between hair shedding and systemic inflammation in COVID-19 pneumonia. Ann Med (2022) 54(1):869–74. doi: 10.1080/07853890.2022.2054026

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Nguyen B, Tosti A. Alopecia in patients with COVID-19: A systematic review and meta-analysis. JAAD Int (2022) 7:67–77. doi: 10.1016/j.jdin.2022.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Monari P, Gualdi G, Bettoni G, Costa R, Ragni G, Zani F, et al. Post-SARS-CoV-2 acute telogen effluvium: An expected complication. J Clin Med (2022) 11(5). doi: 10.3390/jcm11051234

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Wiech M, Chroscicki P, Swatler J, Stepnik D, De Biasi S, Hampel M, et al. Remodeling of T cell dynamics during long COVID is dependent on severity of SARS-CoV-2 infection. Front Immunol (2022) 13:886431. doi: 10.3389/fimmu.2022.886431

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Qi H, Liu B, Wang X, Zhang L. The humoral response and antibodies against SARS-CoV-2 infection. Nat Immunol (2022) 23(7):1008–20. doi: 10.1038/s41590-022-01248-5

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Kent SJ, Khoury DS, Reynaldi A, Juno JA, Wheatley AK, Stadler E, et al. Disentangling the relative importance of T cell responses in COVID-19: Leading actors or supporting cast? Nat Rev Immunol (2022) 22(6):387–97. doi: 10.1038/s41577-022-00716-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: coronavirus, COVID-19, interferon, interferon gamma release assay, QuantiFERON, T cell, SARS-CoV-2

Citation: Fonseca Brito L, Tödter S, Kottlau J, Cermann K, Spier A, Petersen E, Schäfer I, Twerenbold R, Aepfelbacher M, Lütgehetmann M and Stahl FR (2023) Performance of an interferon-γ release assay-based test for cell-mediated immunity to SARS-CoV-2. Front. Immunol. 14:1069968. doi: 10.3389/fimmu.2023.1069968

Received: 17 October 2022; Accepted: 31 January 2023;
Published: 16 February 2023.

Edited by:

Harald H. Kessler, Medical University of Graz, Austria

Reviewed by:

Julia Roider, LMU Munich University Hospital, Germany
Christof Geldmacher, LMU Munich University Hospital, Germany
Hector Bonilla, Stanford Healthcare, United States
Jingang Gui, Beijing Children Hospital, Capital Medical University, China

Copyright © 2023 Fonseca Brito, Tödter, Kottlau, Cermann, Spier, Petersen, Schäfer, Twerenbold, Aepfelbacher, Lütgehetmann and Stahl. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Felix R. Stahl, f.stahl@uke.de

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.