Skip to main content

ORIGINAL RESEARCH article

Front. Pediatr., 14 June 2023
Sec. Pediatric Critical Care

Rapid response events with multiple triggers are associated with poor outcomes in children

\r\nErin M. Kritz,
Erin M. Kritz1,2*Jenilea K. Thomas,Jenilea K. Thomas1,2Nawara S. Alawa,Nawara S. Alawa1,2Elit B. Hadad,Elit B. Hadad1,2Danielle M. GuffeyDanielle M. Guffey1Aarti C. Bavare,\r\nAarti C. Bavare1,2
  • 1Department of Pediatric Critical Care, Baylor College of Medicine, Houston, TX, United States
  • 2Department of Pediatrics, Texas Children’s Hospital, Houston, TX, United States

Objective: We describe the characteristics and outcomes of pediatric rapid response team (RRT) events within a single institution, categorized by reason for RRT activation (RRT triggers). We hypothesized that events with multiple triggers are associated with worse outcomes.

Patients and Methods: Retrospective 3-year study at a high-volume tertiary academic pediatric hospital. We included all patients with index RRT events during the study period.

Results: Association of patient and RRT event characteristics with outcomes including transfers to ICU, need for advanced cardiopulmonary support, ICU and hospital length of stay (LOS), and mortality were studied. We reviewed 2,267 RRT events from 2,088 patients. Most (59%) were males with a median age of 2 years and 57% had complex chronic conditions. RRT triggers were: respiratory (36%) and multiple (35%). Transfer to the ICU occurred after 1,468 events (70%). Median hospital and ICU LOS were 11 and 1 days. Need for advanced cardiopulmonary support was noted in 291 events (14%). Overall mortality was 85 (4.1%), with 61 (2.9%) of patients having cardiopulmonary arrest (CPA). Multiple RRT trigger events were associated with transfer to the ICU (559 events; OR 1.48; p < 0.001), need for advanced cardiopulmonary support (134 events; OR 1.68; p < 0.001), CPA (34 events; OR 2.36; p = 0.001), and longer ICU LOS (2 vs. 1 days; p < 0.001). All categories of triggers have lower odds of need for advanced cardiopulmonary support than multiple triggers (OR 1.73; p < 0.001).

Conclusions: RRT events with multiple triggers were associated with cardiopulmonary arrest, transfer to ICU, need for cardiopulmonary support, and longer ICU LOS. Knowledge of these associations can guide clinical decisions, care planning, and resource allocation.

Introduction

Rapid response teams (RRT) are widely implemented and have demonstrated a positive impact on patient safety and outcomes (17). The goal of RRTs is to manage deteriorating hospitalized patients to decrease out of ICU cardiac arrests and prevent morbidity and mortality (810). It is important to identify patient and RRT event factors which may signal need for transfer to a higher level of care and further critical care interventions (1113).

Adult literature suggests that certain signs and symptoms of clinical instability that trigger RRT events are associated with worse patient outcomes. The nature and number of triggers can guide appropriate resource allocation to prevent further deterioration and adverse outcomes (1418). Most adult studies classify RRT triggers under the categories of respiratory, cardiac, neurological, medical, or other/unknown, with respiratory and cardiac triggers being most common (19). Respiratory triggers have been described as predictive of mortality in pediatric and adult inpatients (1921). Adult studies have found an increase in in-hospital mortality with the number of simultaneous RRT triggers (22). This work expands the pediatric literature to describe and evaluate multiple trigger RRT events with patient outcomes.

We reviewed the triggers of RRT events within a single hospitalization at a large academic pediatric hospital. We hypothesized that RRT events due to multiple triggers are associated with worse outcomes. Early identification of patients at risk for further deterioration at the time of RRT event can guide early resuscitation efforts, aid clinical decision and disposition determination, and improve communication between ICU and acute care providers. For patients with chronic conditions, this can facilitate early discussion regarding disease prognostication and resuscitation status.

Methods

Study design and setting

We conducted a retrospective study of index RRT events over a 3 year-period to compare patient and RRT event characteristics, interventions, and outcomes. The institutional review board approved this study. Our hospital is a large academic pediatric hospital with two community campuses with around 1,000 RR events per year. Our institution has a well-established RRT policy in which the RRT team can be activated by any concerned hospital provider or family member through a centralized number. RRT teams are composed of ICU staff who respond promptly (within 15 min per institutional policy) to RRT activations. Activation criteria (triggers) include any form of respiratory, cardiac, or neurologic decompensation of hospitalized patients who are outside ICU settings. There are no automatic criteria for activation based on patient acuity scores or vital sign parameters. ICU admission is not required for initiation of high-flow nasal cannula within a maximum flow per age, for continuous albuterol therapy, or for patients on home noninvasive ventilation settings. Our central and community campuses have ICUs and follow the same RRT procedures. Further details on our institution's RRT and code team composition, triggers, and activation process are outlined in Supplementary Table S1.

Data collection, variables, and definitions

We collected data from RRT administrative database, electronic medical records, and the Pediatric Health Information System database. Our institutional RRT administrative database includes patient and event details for all RRT events. Patient demographic information was collected as well as clinical data, including primary diagnosis, admission and discharge dates, and primary service. RRT event details and outcomes collected were date and time of event, time to and duration of response, interventions required, trigger(s) for activation, transfer to ICU, additional cardiopulmonary support required within 24 h of transfer, mortality, and cardiopulmonary arrest at time of RRT event. Triggers were categorized and entered into the medical record by the RRT team based upon information provided during handoff by the activating team and assessment of clinical signs and symptoms present during RRT event. Trigger categories are respiratory, cardiac, neurologic, and other based upon standardized definitions at our institution. Events with triggers in more than one organ system were noted as multiple triggers. Chart review by an independent investigator was performed to ensure the original trigger categorization held true. Standardized trigger definitions at our institution are as follows:

Primary diagnosis was used to categorize patients based upon the principal organ system involved. Consistent with the literature, patients were classified as having complex chronic conditions if they had a “medical condition that can be reasonably expected to last at least 12 months (unless death intervenes) and to involve either several different organ systems or one organ system severely enough to require specialty pediatric care” (23, 24). This included patients with chronic conditions, history of prematurity, organ transplant, and/or technological dependence.

Outcome variables reviewed included acute respiratory compromise (ARC, need for bag-mask ventilation or endotracheal intubation) or cardiopulmonary arrest (CPA) during RRT event, transfer to higher level of care, need for advanced cardiopulmonary support (defined as need for mechanical ventilation and/or hemodynamic support within 24 h of RRT event), hospital and ICU length of stay (LOS), and in-hospital mortality.

Statistical analyses

Patient and RRT characteristics and outcomes are summarized using mean with standard deviation, median with interquartile range (IQR), and frequency with percentages. Characteristics and outcomes are compared by triggers using quantile regression, Chi-square test and Fisher's exact test. Simple and multivariable logistic regression was used to assess the association between variables age, gender, primary organ system, complex chronic conditions, RRT trigger, and main campus location and the odds of requiring advanced cardiopulmonary support (need for mechanical ventilation and/or hemodynamic support within 24 h of RRT activation) and odds of mortality. Simple and multivariable quantile regression was used to assess the association between variables age, gender, primary organ system, complex chronic conditions, RRT trigger, and main campus location and the median hospital and ICU LOS. All models adjust for the same factors. The Hosmer-Lemeshow test assessed the goodness of fit of the model and multicollinearity was assessed with the variance inflation factor. There were not goodness of fit or multicollinearity issues found. For simplicity we are presenting the index RRT and index admission per patient to avoid correlated data issues. All analyses are performed using Stata 15.0 (Stata Corp, College Station, TX).

Results

Patient and RR event characteristics

We reviewed a total of 2,267 RRT events in 2,088 patients and included the index 2,088 RRT events in our analysis.Triggers for RRT activation were primarily respiratory [758 (36.3%)] and multiple [739 (35.4%)], with the remainder being cardiac [236 (11.3%)], neuro [218 (10.4%)], and other [137 (6.6%)]. Table 1 presents the patient characteristics by RR triggers.

TABLE 1
www.frontiersin.org

Table 1. Patient characteristics by RRT trigger.

Outcomes

Transfer to the ICU occurred after 1,468 events (70%). Median hospital LOS was 11 days (IQR 9–12 days) and median ICU LOS was 1 day (IQR 0–1 days). Need for advanced cardiopulmonary support within 24 h was noted in 291 events (14%): mechanical ventilation [196 (67%)], vasoactive medications [34 (12%)], and mixed cardiac and respiratory support [61 (21%)]. Hospital mortality for the cohort was 4.1% (85), with 61 (2.9%) of patients having CPA during the event.

Outcomes analyses

Multiple trigger RRT events had higher rate of transfer to ICU (559 patient events; OR 1.48; p < 0.001) and higher rate of CPA (34 events, OR 2.36; p = 0.001) than RRT events with any other types of triggers. Hospital LOS was longer for patients with chronic medical conditions. Type of trigger, however, was not associated with length of stay after adjusting for confounders of age, gender, primary organ system, and chronic condition. ICU LOS is longest in multiple trigger events (2 vs. 1 days; IQR 0–5 d; p < 0.001). All categories of triggers have lower odds of need for advanced cardiopulmonary support than multiple triggers (unadjusted OR 1.73; CI: 1.29–2.32; p < 0.001), Figure 1. After adjusting for confounders, multiple triggers were associated with higher odds for advanced cardiopulmonary support compared to other triggers: respiratory (adjusted OR, 1.68; CI: 1.25–2.27; p = 0.001), cardiac (adjusted OR: 1.49; CI: 0.97–2.29; p < 0.001), neuro (adjusted OR: 1.69; CI: 1.07–2.68; p = 0.03), and other triggers (adjusted OR: 5.13; CI: 2.19–12.01; p < 0.001). Patients with chronic conditions were found to have higher odds of mortality (OR, 6.7; CI 2.34–19.39; p < 0.001). After adjusting for confounders, trigger was not independently associated with mortality. Table 2 details RRT event outcomes by triggers.

FIGURE 1
www.frontiersin.org

Figure 1. Type of advanced cardiopulmonary support by trigger, p = <0.001. Type of advanced cardiopulmonary support required depicted by RRT trigger. RRT, rapid response team.

TABLE 2
www.frontiersin.org

Table 2. Patient Outcomes by RR Trigger.

Discussion

There is limited pediatric data available regarding RRT triggers and association with outcomes. Identification of at-risk patients often includes the use of triggers which are informed by aggregate pediatric early warning scores, monitored vital signs, flagging of diagnostic risk factors, as well as medical team and parental concern regarding clinical status. To our knowledge, this is the largest cohort in pediatric patients used to assess the association of RRT triggers with outcomes (2531). Our results support our hypothesis that RRT trigger was associated with several outcomes. Children with multiple triggers for RRT events had longer ICU LOS, higher rates of cardiopulmonary arrest, and need for advanced cardiopulmonary support, which may signal a sicker patient who requires advanced ICU interventions and longer ICU stay. As initiation and use of NIPPV can be somewhat subjective and dependent on specific practices to each center, we did a separate sub analysis of patients who required mechanical ventilation and/or hemodynamic support; and indeed, patients with multiple-trigger RRT events did more often require this type of support. We felt these represented more objective ICU-level interventions which could be interpreted and applied to a variety of centers. Although trigger type was not associated with mortality, the association with other significant outcomes could impact clinical decisions, care planning, and resource allocation.

As trigger is known at the time of RRT event, this may offer a means of prognostication and allocation of resources. Patients with multiple triggers may benefit from more thorough evaluation by the ICU team with a lower threshold to transfer to higher level of care. This categorization may also aid expectations regarding length of stay and level of monitoring in the ICU or acute care ward, as well as higher vigilance by the clinical team for signs of further deterioration.

The main limitations of this study relate to its retrospective nature. Outcomes data for a few patients was either incomplete or unavailable. Given the overall rarity of catastrophic outcomes in the pediatric population, it was difficult to demonstrate association with some outcomes. Finally, this work was conducted at a single center with processes and procedures that may be unique to our institution, thereby limiting generalizability. Nevertheless, we believe this is an expanded approach to RRT event analysis in pediatric patients and offers potential for prognostication regarding outcomes at the time of event.

Conclusion

We confirmed that RRT events with multiple triggers were associated with increased ICU length of stay, increased frequency of cardiopulmonary arrest outside of the ICU, and need for a higher level of care. Regular review of RRT events and outcomes can inform a more targeted use of triggers to better prognosticate patient outcomes and effective allocation of resources.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

EK conceptualized and designed the study, drafted, reviewed, and revised the manuscript; NA and EH designed the study, collected data, and drafted the initial manuscript; DM designed the data collection instruments, collected data, analyzed the data, and reviewed and revised the manuscript; JT critically reviewed and revised the manuscript; AB conceptualized the study, supervised data collection, and critically reviewed and revised the manuscript. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2023.1208873/full#supplementary-material

References

1. Jones DA, DeVita MA, Bellomo R. Rapid-response teams. N Engl J Med. (2011) 365(2):139–46. doi: 10.1056/NEJMra0910926

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Felner K, Smith RL. Rapid-response teams. N Engl J Med. (2011) 365(14):1355–6; author reply 1356–7. doi: 10.1056/NEJMc1109223

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Manthous CA. Rapid-response teams. N Engl J Med. (2011) 365(14):1356; author reply 1356–7. doi: 10.1056/NEJMc1109223

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 lives campaign: setting a goal and a deadline for improving health care quality. JAMA. (2006 ) 295(3):324–7. doi: 10.1001/jama.295.3.324

PubMed Abstract | CrossRef Full Text | Google Scholar

5. White K, Scott IA, Bernard A, McCulloch K, Vaux A, Joyce C, et al. Patient characteristics, interventions and outcomes of 1151 rapid response team activations in a tertiary hospital: a prospective study. Intern Med J. (2016) 46(12):1398–406. doi: 10.1111/imj.13248

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Fernando SM, Reardon PM, Scales DC, Murphy K, Tanuseputro P, Heyland DK, et al. Prevalence, risk factors, and clinical consequences of recurrent activation of a rapid response team: a multicenter observational study. J Intensive Care Med. (2019) 34(10):782–9. doi: 10.1177/0885066618773735

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review and meta-analysis. Crit Care. (2015) 19(1):254. doi: 10.1186/s13054-015-0973-y

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Jones D, Opdam H, Egi M, Goldsmith D, Bates S, Gutteridge G, et al. Long-term effect of a medical emergency team on mortality in a teaching hospital. Resuscitation. (2007) 74(2):235–41. doi: 10.1016/j.resuscitation.2006.12.007

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Sharek PJ, Parast LM, Leong K, Coombs J, Earnest K, Sullivan J, et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children’s hospital. JAMA. (2007) 298(19):2267–74. doi: 10.1001/jama.298.19.2267

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Beitler JR, Link N, Bails DB, Hurdle K, Chong DH. Reduction in hospital-wide mortality after implementation of a rapid response team: a long-term cohort study. Crit Care. (2011) 15(6):R269. doi: 10.1186/cc10547

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Solomon RS, Corwin GS, Barclay DC, Quddusi SF, Dannenberg MD. Effectiveness of rapid response teams on rates of in-hospital cardiopulmonary arrest and mortality: a systematic review and meta-analysis. J Hosp Med. (2016) 11(6):438–45. doi: 10.1002/jhm.2554

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Meulmester KM, Jaimon N, Bavare AC. Repeat rapid response events in children: characteristics and outcomes. Pediatr Crit Care Med. (2018) 19(4):361–8. doi: 10.1097/PCC.0000000000001457

PubMed Abstract | CrossRef Full Text | Google Scholar

13. McKelvie B, McNally JD, Chan J, Momoli F, Ramsay C, Lobos AT. Increased mortality and length of stay associated with medical emergency team review in hospitalized pediatric patients: a retrospective cohort study. Pediatr Crit Care Med. (2017) 18(6):571–9. doi: 10.1097/PCC.0000000000001164

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Foraida MI, DeVita MA, Braithwaite RS, Stuart SA, Brooks MM, Simmons RL. Improving the utilization of medical crisis teams (condition C) at an urban tertiary care hospital. J Crit Care. (2003) 18(2):87–94. doi: 10.1053/jcrc.2003.50002

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Jones D, Bellomo R, DeVita MA. Effectiveness of the medical emergency team: the importance of dose. Crit Care. (2009) 13(5):313. doi: 10.1186/cc7996

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response systems as a patient safety strategy: a systematic review. Ann Intern Med. (2013) 158(5, pt 2):417–25. doi: 10.7326/0003-4819-158-5-201303051-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bonafide CP, Localio AR, Roberts KE, Nadkarni VM, Weirich CM, Keren R. Impact of rapid response system implementation on critical deterioration events in children. JAMA Pediatr. (2014) 168(1):25–33. doi: 10.1001/jamapediatrics.2013.3266

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Panesar R, Polikoff LA, Harris D, Mills B, Messina C, Parker MM. Characteristics and outcomes of pediatric rapid response teams before and after mandatory triggering by an elevated Pediatric Early Warning System (PEWS) score. Hosp Pediatr. (2014) 4(3):135–40. doi: 10.1542/hpeds.2013-0062

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lyons PG, Edelson DP, Carey KA, Twu NM, Chan PS, Peberdy MA, et al. Characteristics of rapid response calls in the United States: an analysis of the first 402,023 adult cases from the get with the guidelines resuscitation-medical emergency team registry. Crit Care Med. (2019) 47(10):1283–9. doi: 10.1097/CCM.0000000000003912

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Smith RJ, Santamaria JD, Faraone EE, Holmes JA, Reid DA. Rapid response team diagnoses: frequencies and related hospital mortality. Crit Care Resusc. (2017) 19(1):71–80.28215135

PubMed Abstract | Google Scholar

21. Meulmester KM, Coss-Bu JA, Meskill SD, Wakefield BM, Moore RH, Vachani JG, et al. Characteristics and outcomes of pediatric rapid response with a respiratory trigger. Hosp Pediatr. (2021) 11(8):806–7. doi: 10.1542/hpeds.2020-004630

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Shappell C, Snyder A, Edelson DP, Churpek MM. Predictors of in-hospital mortality after rapid response team calls in a 274 hospital nationwide sample. Crit Care Med. (2018) 46(7):1041–8. doi: 10.1097/CCM.0000000000002926

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Feudtner C, Christakis DA, Connell FA. Pediatric deaths attributable to complex chronic conditions: a population-based study of Washington state, 1980-1997. Pediatrics. (2000) 106(1 pt 2):205–9. doi: 10.1542/peds.106.S1.205

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Feudtner C, Feinstein JA, Zhong W, Hall M, Dai D. Pediatric complex chronic conditions classification system version 2: updated for ICD-10 and complex medical technology dependence and transplantation. BMC Pediatr. (2014) 14:199. doi: 10.1186/1471-2431-14-199

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA. (2008) 300(21):2506–13. doi: 10.1001/jama.2008.715

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Brilli RJ, Gibson R, Luria JW, Wheeler TA, Shaw J, Linam M, et al. Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit. Pediatr Crit Care Med. (2007) 8(3):236–46. doi: 10.1097/01.PCC.0000262947.72442.EA

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Fischer CP, Bilimoria KY, Ghaferi AA. Rapid response teams as a patient safety practice for failure to rescue. JAMA. (2021) 326(2):179–80. doi: 10.1001/jama.2021.7510

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Sosa T, Ferris S, Frese C, Hacker D, Dewan M, Brady PW. Comparing two proximal measures of unrecognized clinical deterioration in children. J Hosp Med. (2020) 15(11):673–6. doi: 10.12788/jhm.3515

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Hussain FS, Sosa T, Ambroggio L, Gallagher R, Brady PW. Emergency transfers: an important predictor of adverse outcomes in hospitalized children. J Hosp Med. (2019) 14(8):482–5. doi: 10.12788/jhm.3219

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Sosa T, Galligan MM, Brady PW. Clinical progress note: situation awareness for clinical deterioration in hospitalized children. J Hosp Med. (2022) 17:199–202. doi: 10.1002/jhm.2774

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Acorda DE, Bracken J, Abela K, Ramsey-Coleman J, Stutts A, Kritz E, et al. Longitudinal evaluation of pediatric rapid response system with realist evaluation framework. Jt Comm J Qual Patient Saf. (2022) 48(4):196–204. doi: 10.1016/j.jcjq.2022.01.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: pediatric outcomes, rapid response, pediatric ICU, critical deterioration, cardiopulmonary arrest (cpa), triggers

Citation: Kritz EM, Thomas JK, Alawa NS, Hadad EB, Guffey DM and Bavare AC (2023) Rapid response events with multiple triggers are associated with poor outcomes in children. Front. Pediatr. 11:1208873. doi: 10.3389/fped.2023.1208873

Received: 19 April 2023; Accepted: 24 May 2023;
Published: 14 June 2023.

Edited by:

Joris Lemson, Radboud University, Netherlands

Reviewed by:

Joris Fuijkschot, Radboud University Medical Centre, Netherlands
Mary E Hartman, Washington University in St. Louis, United States

© 2023 Kritz, Thomas, Alawa, Hadad, Guffey and Bavare. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Erin M. Kritz erin.kritz@bcm.edu

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.