Skip to main content

SYSTEMATIC REVIEW article

Front. Med., 24 September 2020
Sec. Infectious Diseases – Surveillance, Prevention and Treatment
This article is part of the Research Topic Coronavirus Disease (COVID-19): Pathophysiology, Epidemiology, Clinical Management and Public Health Response View all 400 articles

The Pipeline of Therapeutics Testing During the Emergency Phase of the COVID-19 Outbreak

  • 1National Center for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy
  • 2Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy
  • 3Department of Sense Organ, Sapienza University of Rome, Rome, Italy
  • 4Unit of Neurosurgery, IRCSS Fondazione Cà Granda Ospedale Maggiore Policlinico, University of Milan, Milan, Italy
  • 5Department of Radiology, ASTT Grande Ospedale Metropolitano Niguarda, University of Milan, Milan, Italy
  • 6Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy
  • 7Geriatric Unit, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy
  • 8Department of Experimental Medicine, Sapienza University of Rome, Rome, Italy

The coronavirus disease 19 (COVID-19) pandemic poses a serious threat to the sustainability of healthcare systems and is currently having a significant effect on living conditions worldwide. No therapeutic agent has yet proven to be effective for the treatment of COVID-19. The management of this disease currently relies on supportive care and the off-label and compassionate use of antivirals and immunomodulators. Nevertheless, there has been a great worldwide effort to progress research and test the efficacy and safety/tolerability profiles of numerous candidate agents that may positively affect the various clinical syndromes associated with COVID-19. In parallel, vaccination and chemoprophylaxis strategies are being investigated. This article provides a summary of interventional studies targeting COVID-19 during the emergency phase of the outbreak to broadly inform clinicians and researchers on what happened and what they can expect in upcoming months. The clinicaltrials.gov database and the European Union (EU) Clinical Trials Register were investigated on March 31, 2020, to identify all ongoing phase 1–4 research protocols testing pharmacological interventions targeting SARS-CoV-2 infection and/or clinical syndromes associated with COVID-19. Overall, six phase 1, four phase 1-2, 14 phase 2, ten phase 2-3, 19 phase 3, and nine phase 4 studies were identified, and the features of these studies are described in the present review. We also provide an updated overview of the change overtime in the pipeline following this emergency phase and based on the current epidemiology of the COVID-19 pandemic.

Introduction

The coronavirus disease 19 (COVID-19) pandemic has been caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). It poses a serious threat to the sustainability of healthcare systems, with substantial effects on living conditions worldwide. As of April 3, 2020, more than one million COVID-19 cases and around 53,000 deaths have been calculated in 181 countries worldwide (1). In parallel, nearly half of the global population is currently in lockdown.

To date, no therapeutic compound has been proven to be effective for the treatment of COVID-19. In the initial emergency phase of the outbreak, therapeutic management of affected individuals relied on supportive care (2, 3) and on the off-label and compassionate use of a variety of antiviral (e.g., lopinavir/ritonavir, remdesivir, favipiravir) and/or immunomodulator (e.g., chloroquine, hydroxychloroquine, anti-IL-6 inhibitors, steroids) drugs, the efficacy of which had not then been demonstrated (4, 5). Moreover, their safety and tolerability profiles in patients with COVID-19 remains to be clarified (4, 6).

In this pandemic scenario, a great deal of effort is currently being devoted to the identification of novel therapies and prophylactic strategies, with new research protocols registered internationally every week (if not daily) (7). Moreover, the urgent need to move this field forward in response to this ongoing outbreak needs to be counterbalanced by ensuring that the products under investigation are evaluated through scientifically and ethically appropriate studies (8). There are challenging time-frames connected to the process of developing new therapeutic strategies against COVID-19 or repositioning existing compounds with plausible modifying effects on the disease. The clinical course of patients is not yet fully elucidated (9), and there is incomplete data on the underlying pathophysiological mechanisms (10) and potential therapeutic targets.

In this article, we provide a summary of the interventional studies that have been conducted worldwide to test the efficacy and/or safety/tolerability of pharmacological compounds against COVID-19 in the emergency phase of the pandemic.

Methods

Data Source

Two databases, the clinicaltrials.gov database and the European Union (EU) Clinical Trials Register, constituted the reference sources for the present study. Clinicaltrials.gov is a web-based resource maintained by the US National Library of Medicine and the National Institute of Health that provides information on publicly and privately supported clinical studies. Registration on this database is mandatory for all clinical investigations of any US Food and Drug Administration (FDA)-regulated drug or medical device. However, it also represents a repository for the vast majority of clinical trial protocols conducted worldwide. EU Clinical Trial Register gathers information on ongoing authorized interventional studies in the EU and the European Economic Area (EEA) that are registered in the EU Drug Regulation Authorities Clinical Trials Database (EudraCT).

Search Strategy

The databases were investigated on March 31, 2020, using the following search terms: “COVID-19” OR “SARS-CoV-2” OR “2019 novel coronavirus” OR “2019-nCoV” OR “severe acute respiratory syndrome coronavirus 2” OR “coronavirus.” In clinicaltrials.gov, the advanced search function was used to restrict the search to: (i) interventional studies (STUDY TYPE); (ii) “recruiting,” “enrolling by invitation,” and “active not recruiting” protocols (STATUS: RECRUITMENT); and (iii) phase 1, phase 2, phase 3, phase 4 studies (PHASE).

Two reviewers (L.T. and G.R.) screened the identified protocols to remove duplicates and verify the fulfillment of the following predefined inclusion criteria: (1) targeting SARS-CoV-2 infection and/or clinical syndromes associated with COVID-19; and (2) testing the efficacy and/or safety/tolerability of pharmacological interventions. Studies investigating novel medical devices or diagnostic tools were not considered in the present analysis. Disagreements in the selection were solved by consensus, involving two additional reviewers (M.C. and V.R.). The flow chart in Figure 1 illustrates the process of protocols' selection.

FIGURE 1
www.frontiersin.org

Figure 1. Flow-chart describing the selection of protocols. MERS, Middle East Respiratory Syndrome.

Data Extraction

The following data were abstracted by three authors (F.T., Ga.R., and Gi.R.) from the selected protocols: NCT (the unique identification code assigned by clinicaltrials.gov) and/or EudraCT codes; study phase; allocation and masking procedures; tested compound(s); way of administration; mechanism of action; primary outcome measure(s); expected primary completion date; expected number of participants; age range of participants; targeted COVID-19 related condition; sponsor; and location.

Clinical syndromes associated with COVID-19 were coded according to the classifications provided by the World Health Organization (WHO) (i.e., mild illness, pneumonia, severe pneumonia, acute respiratory distress syndrome [ARDS], sepsis, and septic shock) (2). When the WHO classification could not be applied or was not specified, the targeted conditions were classified according to the definitions provided in the protocol.

Results

Search Results

A total of 74 protocols were identified through a structured search of the two adopted databases (n = 57 on clinicaltrials.gov and n = 17 on EU Clinical Trials Register). After the removal of six duplicates, six additional protocols were excluded because not targeting COVID-19. Specifically, five of them were focused on the Middle East Respiratory Syndrome and one on other bacterial or viral infections. Thus, 62 protocols were ultimately retained. The reviewers reported a >90% agreement in the selection process.

Characteristics of the Selected Protocols

Clinical trials involving new drugs are commonly classified into four phases, with some individual trials encompassing more than one phase (e.g., combined phase 1–2). Overall, six phase 1, four phase 1–2, 14 phase 2, ten phase 2–3, 19 phase 3, and nine phase 4 studies were identified. Their detailed characteristics are presented in Tables 13.

TABLE 1
www.frontiersin.org

Table 1. Characteristics of the selected phase 1, phase 1-2, phase 2, and phase 2-3 protocols targeting COVID-19-related conditions.

Most trials were conducted in China (n = 30), followed by the US (n = 10), Italy (n = 8), Germany (n = 6), France (n = 6), Spain (n = 5), and Korea (n = 5) (Figure 2). Seven trials involved international networks of clinical sites, whereas 55 were run in single countries; 35% of studies (n = 22) are multicentric. The majority of studies were funded by non-commercial research institutions (e.g., universities, hospitals, foundations, institutes) while only 11 were sponsored by the biopharma industry. Protocols had a varying duration and are expected to be completed (in terms of primary completion) between April 2020 and July 2023.

FIGURE 2
www.frontiersin.org

Figure 2. Geographic location of the selected protocols.

Forty-five trials had a randomized design, mostly relying on a parallel assignment of participants. Masking procedures were instead adopted by less than half of the trials (n = 26), with nine studies reporting quadruple masking involving both the participants, investigators, care providers, and outcomes assessors. Placebo or standard care were used as comparators in 27 studies, and 14 use one or more active comparators, whereas nine compared different regimens (i.e., dosages and/or duration) of the same treatment.

Tested Interventions

Most protocols (n = 32) investigated the efficacy and/or safety profiles of compounds that are expected to act as immune system modulators in COVID-19 associated conditions (Figure 3). These compounds included vaccines (n = 5), cell-based therapies (n = 6; e.g., mesenchymal stem cells, natural killer cells), antimalarial drugs (n = 9; e.g., chloroquine and hydroxychloroquine), corticosteroids (n = 4), interleukin inhibitors, and interferons. Twenty-two studies have been testing antiviral agents such as antiretroviral protease inhibitors (e.g., darunavir, lopinavir, ritonavir), neuraminidase inhibitors (e.g., oseltamivir), nucleotide analogs (e.g., remdesivir), and broad-spectrum antivirals. The remaining trials were designed to investigate other potential adjuvant therapies such as nitric oxide, antioxidants, phosphodiesterase inhibitors. Finally, seven studies have been evaluating the combinations of substances with both immunomodulant and antiviral properties.

FIGURE 3
www.frontiersin.org

Figure 3. Pharmacological interventions against COVID-19 tested in the emergency phase of the outbreak according to the study phase and mechanism of action. IFN, interferon; MSCs, mesenchymal stem cells; NK, natural killers.

Most of the selected primary outcome measures referred to clinical endpoints (e.g., mortality rates, clinical improvement/remission, hospital discharge, intensive care unit admission, ventilation-free days). A sizeable proportion of studies (n = 24) incorporated laboratory (e.g., viral clearance/load) or radiological (e.g., change of pneumonia severity on X-ray or CT) changes as primary endpoints. A residual number of trials (n = 7) were instead primarily aimed at exploring the safety and tolerability profiles of the tested interventions.

Targeted Conditions

A total of 41,110 participants will tentatively be recruited in the selected protocols, with sample sizes widely ranging between five and 6,800 subjects.

The entire clinical spectrum of COVID-19, ranging from infection with mild symptoms to sepsis complicated by shock, was targeted by the studies in the emergency phase of the COVID-19 outbreak. It also planned to recruit healthy subjects or individuals exposed to higher risk (e.g., healthcare providers or household contact).

Early phase studies (Table 1) preliminarily tested the tolerated dose, the safety, and efficacy of candidate agents in small representative groups. The target population was composed by healthy subjects (n = 4), individuals with laboratory confirmed infection without a clear WHO definition of the clinical syndrome (n = 6), patients with pneumonia ranging from mild to severe (n = 19), and more severe/critical clinical syndromes including acute respiratory distress syndrome (ARDS) and sepsis (n = 14). In the late phases studies (Tables 2, 3) that are testing on a large scale those agents with documented safety and evidence of preliminary efficacy in the earlier phases, the target participants were largely represented by healthy or at-risk subjects with infection (n = 7), patients with mild/severe pneumonia (n = 20), and patients with more severe/critical clinical syndrome (n = 10).

TABLE 2
www.frontiersin.org

Table 2. Characteristics of the selected phase 3 protocols targeting COVID-19-related conditions.

TABLE 3
www.frontiersin.org

Table 3. Characteristics of the selected phase 4 protocols targeting COVID-19-related conditions.

Discussion

Although the first COVID-19 cases were reported just 4 months ago (11), there has been an unprecedented response from the international community. The findings of several interventional studies have already been disseminated (5, 12). Encouragingly, a relevant number of clinical trials have explored safe and effective therapeutics to face the pandemic, enrolling individuals with COVID-19 worldwide, and some of these trials will publish the results as early as in the next few weeks/months. This emerging evidence will largely be concerned with hard outcomes such as mortality (adopted as the primary endpoint in ten studies), access to intensive care units, clinical remission, and will therefore have profound clinical implications.

This review provides an overview of studies in the emergency phase of the outbreak, utilizing the two most common open access protocol registries in the US and Europe, with the aim of informing clinicians and researchers on what they can expect in the upcoming months. Of note, is the fact that we restricted our focus to only two clinical trial registries and we are aware that this might potentially underestimate the current situation. When a broader search is conducted by including most of the existing national and international databases, the number of ongoing studies is much higher and needs to be constantly updated (13). Accordingly, coalitions/networks have recently been launched to provide frequently updated resources (e.g., living systematic reviews) summarizing the characteristics of research protocols targeting COVID-19 (7, 13). These initiatives are particularly welcomed, as they potentially allow for the coordination of a multinational research effort and better allocation of the available research resources.

As expected, interventional studies were largely performed and promoted in those countries where the outbreak has already significantly affected the community and the healthcare system. The inclusion criteria of the studies was designed to target the entire spectrum of clinical syndromes associated with COVID-19 at the time the study was conducted, namely asymptomatic status, mild illness, pneumonia, ARDS, and septic complications. The opportunity to include the clinical struggles for different categories of patients was also implemented. Several trials were instead focused on the vaccination and chemoprophylaxis of healthy individuals. Two studies were specifically dedicated to health care providers, consistently with their established vulnerability in the COVID-19 pandemic (14). These studies are very much needed, as in some countries the number of healthcare providers with infection is rapidly increasing due to a shortage of personal protective equipment, in parallel with the high demand for care that usually occurs during a pandemic (15). Currently, slowing the spread of the SARS-CoV-2 relies on measures of social distancing and recommended changes to lifestyle and behavior that have unmeasurable consequences on the life of individuals and communities, not to mention the economic crisis that countries face. In light of this, it is pivotal to cooperate and optimize the effort for a common solution starting from the systematic recruitment of patients to complete the ongoing trials.

Based on the registered information, some protocols will probably provide proof-of-concept evidence supporting the design of large-scale clinical trials. Conversely, some of the ongoing phase 3 randomized controlled trials and phase 4 post marketing studies seem already adequately informed to be able to draw either positive or negative conclusions on the efficacy and safety/tolerability of pharmacological compounds with different mechanisms of action. Of note, is the fact that some trials are adopting adaptive designs, allowing them to rapidly accept or reject multiple experimental therapies, which is especially promising in the current outbreak scenario (4).

The major limitation of our study is related to the extremely dynamic evolution of knowledge on the topic. As mentioned, an incredible number of trials have been proposed on COVID during the past weeks and it is likely that this number will rapidly and exponentially increase in the next months, especially given the more consistent dissemination of the coronavirus in different regions of the world. In this regard, since April 1, 2020, 585 new protocols have been registered on the clinicaltrials.gov database (search updated to August 18, 2020) with an expected overall number of around 375,000 participants. As compared with the emergency phase, a greater proportion of phase 1 and 2 studies are currently active (70.2 vs. 54.8%). An increase in the percentage of industry-funded trials (34.2 vs. 17.7%) and of studies adopting a randomized design (85.3 vs. 72.6%) has been observed. As expected by the changes that have occurred in the epidemiology of COVID-19, the US and Europe persist as the main recruiting sites while centers in South America, India, and Africa have recently started to contribute. It is noteworthy that, due to a better understanding of the pathophysiological mechanisms of the disease (10), there are a relevant number of novel compounds, mostly acting as immunomodulators, that are being tested (e.g., ruxolitinib, colchicine, heparins, mavirilumab, ivermectin). These were not present nor in the pipeline at the end of March 2020. Moreover, 47 protocols are currently investigating the efficacy and safety profiles of vaccines whereas 67 focused on convalescent plasma therapies. As of August 18, 2020, the (negative) findings of four of the studies that started in the emergency phase have already been published (i.e., three testing remdesivir [NCT04292899, NCT04292730, NCT04257656] and one testing hydroxychloroquine [NCT04308668]) (1619).

In conclusion, the present analysis provides an account for researchers and clinicians for them to understand present research and envision the future of therapeutics testing for the management of the COVID-19 pandemic.

Data Availability Statement

All datasets presented in this study are included in the article.

Author Contributions

MCa and VR: study design, data analysis, and writing of the manuscript. GRe: study design, data collection, and drafting of the manuscript. FT, GRic, LT, GRis, and AA: data collection. GB, MCe, and NV: data interpretation and drafting of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

MCa is supported by a research grant of the Italian Ministry of Health for the project Dementia in immigrants and ethnic minorities living in Italy: clinical-epidemiological aspects and public health perspectives (ImmiDem) (GR-2016-02364975). VR was supported by the Scientific Independence of Young Researchers Program (RBSI14HNVT), Italian Ministry of Education, University and Research (MIUR), Rome, Italy.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank Marco Moroni for the realization of Figure 3. We thank Dr. Giuseppe Traversa (National Centre for Drug Research and Evaluation, Italian National Institute of Health) for revising the manuscript for intellectual content.

References

1. Johns Hopkins Coronavirus Resource Center. COVID-19 Map. Available online at: https://coronavirus.jhu.edu/map.html (accessed April 3, 2020)

Google Scholar

2. World Health Organization. Clinical management of severe acute respiratory infection when COVID-19 is suspected. Pediatria Medycyna Rodzinna. (2020) 16:9–26. doi: 10.15557/PiMR.2020.0003

CrossRef Full Text | Google Scholar

3. Poston JT, Patel BK, Davis AM. Management of critically Ill adults with COVID-19. JAMA. (2020). doi: 10.1001/jama.2020.4914. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kalil AC. Treating COVID-19—Off-label drug use, compassionate use, and randomized clinical trials during pandemics. JAMA. (2020). doi: 10.1001/jama.2020.4742. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Cao B, Wang Y, Wen D, Liu W, Wang J, Fan G, et al. A Trial of Lopinavir-Ritonavir in Adults Hospitalized with Severe Covid-19. N Engl J Med. (2020) 382:1787–1799. doi: 10.1056/NEJMc2008043

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Russell CD, Millar JE, Baillie JK. Clinical evidence does not support corticosteroid treatment for 2019-nCoV lung injury. Lancet. (2020) 395:473–5. doi: 10.1016/S0140-6736(20)30317-2

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Maguire BJ, Guérin PJ. A living systematic review protocol for COVID-19 clinical trial registrations. Wellcome Open Res. (2020) 5:60. doi: 10.12688/wellcomeopenres.15821.1

PubMed Abstract | CrossRef Full Text | Google Scholar

8. European Medicines Agency. Points to Consider on Implications of Coronavirus Disease (COVID-19) on Methodological Aspects of Ongoing Clinical Trials. Amsterdam: European Medicines Agency (2020).

Google Scholar

9. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. (2020) 395:1054–62. doi: 10.1016/S0140-6736(20)30566-3

CrossRef Full Text | Google Scholar

10. Kabir MT, Uddin MS, Hossain MF, Abdulhakim JA, Alam MA, Ashraf GM, et al. nCOVID-19 pandemic: from molecular pathogenesis to potential investigational therapeutics. Front Cell Dev Biol. (2020) 8:616. doi: 10.3389/fcell.2020.00616

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. (2020) 395:497–506. doi: 10.1016/S0140-6736(20)30183-5

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Chen Z, Hu J, Zhang Z, Jiang S, Han S, Yan D, et al. Efficacy of hydroxychloroquine in patients with COVID-19: results of a randomized clinical trial. medRxiv. doi: 10.1101/2020.03.22.20040758. [Epub ahead of print].

CrossRef Full Text | Google Scholar

13. COVID-19 Clinical Research Coalition. Global coalition to accelerate COVID-19 clinical research in resource-limited settings. Lancet. (2020) 395:1322–5. doi: 10.1016/S0140-6736(20)30798-4

PubMed Abstract | CrossRef Full Text | Google Scholar

14. The Lancet. COVID-19: protecting health-care workers. Lancet. (2020) 395:922. doi: 10.1016/S0140-6736(20)30644-9

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Ranney ML, Griffeth V, Jha AK. Critical supply shortages — the need for ventilators and personal protective equipment during the Covid-19 pandemic. N Engl J Med. (2020) 382:e41. doi: 10.1056/NEJMp2006141

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Goldman JD, Lye DCB, Hui DS, Marks KM, Bruno R, Montejano R, et al. Remdesivir for 5 or 10 days in patients with severe Covid-19. N Engl J Med. (2020). doi: 10.1056/NEJMoa2015301. [Epub ahead of print].

CrossRef Full Text | Google Scholar

17. Spinner CD, Gottlieb RL, Criner G, Arribas López JR, Cattelan AM, Viladomiu AS, et al. Effect of remdesivir vs standard care on clinical status at 11 days in patients with moderate COVID-19: a randomized clinical trial. JAMA. (2020). doi: 10.1001/jama.2020.16349. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Wang Y, Zhang D, Du G, Du R, Zhao J, Jin Y, et al. Remdesivir in adults with severe COVID-19: a randomised, double-blind, placebo-controlled, multicentre trial. Lancet. (2020) 395:1569–78. doi: 10.1016/S0140-6736(20)31022-9

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Skypper CP, Pastick KA, Engen NW, Bangiwala AS, Abassi M, Lofgren SM, et al. Hydroxychloroquine in nonhospitalized adults with early COVID-19: a randomized trial. Ann Intern Med. (2020) 16:M20–4207. doi: 10.7326/M20-4207

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: COVID-19, SARS-CoV-2, clinical trials, antivirals, immunomodulators, research protocols, drug development

Citation: Canevelli M, Remoli G, Trentin F, Riccardi G, Tariciotti L, Risoleo G, Ancidoni A, Bruno G, Cesari M, Vanacore N and Raparelli V (2020) The Pipeline of Therapeutics Testing During the Emergency Phase of the COVID-19 Outbreak. Front. Med. 7:552991. doi: 10.3389/fmed.2020.552991

Received: 17 April 2020; Accepted: 01 September 2020;
Published: 24 September 2020.

Edited by:

Mohan Jyoti Dutta, Massey University, New Zealand

Reviewed by:

Xiaojiong Jia, Harvard Medical School, United States
Mohamed M. Abdel-Daim, Suez Canal University, Egypt

Copyright © 2020 Canevelli, Remoli, Trentin, Riccardi, Tariciotti, Risoleo, Ancidoni, Bruno, Cesari, Vanacore and Raparelli. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Marco Canevelli, bWFyY28uY2FuZXZlbGxpJiN4MDAwNDA7aXNzLml0

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.