Nina M. Kaper
Geerte G. J. Ramakers1- 1Department of Otolaryngology–Head and Neck Surgery, University Medical Center Utrecht, Utrecht, Netherlands
- 2Department of Otolaryngology, Jeroen Bosch Hospital, s'-Hertogenbosch, Netherlands
- 3Department of Social Dentistry, Academic Center for Dentistry Amsterdam, University of Amsterdam, VU University, Amsterdam, Netherlands
Background: We wanted to asses and characterize the volume of Otolaryngology publications on clinical research, published in major journals.
Methods and Material: To assess volume and study type of clinical research in Otolaryngology we performed a literature search in high impact factor journals. We included 10 high impact factor Otolaryngology journals and 20 high impact factor medical journals outside this field (2011). We extracted original publications and systematic reviews from 2010. Publications were classified according to their research question, that is therapy, diagnosis, prognosis or etiology.
Results: From Otolaryngology journals (impact factor 1.8 to 2.8) we identified 694 (46%) publications on original observations and 27 (2%) systematic reviews. From selected medical journals (impact factor 6.0 to 101.8) 122 (2%) publications related to Otolaryngology, 102 (83%) were on original observations and 2 (0.04%) systematic reviews. The most common category was therapy (40%).
Conclusion: Half of publications in Otolaryngology concerns clinical research, which is higher than other specialties. In medical journals outside the field of Otolaryngology, a small proportion (2%) of publications is related to Otolaryngology. Striking is that systematic reviews, which are considered high level evidence, make up for only 2% of publications. We must ensure an increase of clinical research for optimizing medical practice.
Introduction
Clinicians strive to provide evidence-based patient care (1). According to the principles of evidence-based medicine (EBM), they should evaluate all available research for the best evidence and combine this with their experience and patients' preferences (1, 2). Therefore, clinicians are in need of publications reporting on health outcomes in patients, that provide answers to clinical research questions. These studies are addressed to as clinical research and have therefore a direct possibility to influence clinical practice (1, 2). Other research types, such as biological experiments or individual clinician's experiences can also be important, but their impact on clinical practice is often limited (2).
We initiated this study because we wanted to asses and characterize the volume of Otolaryngology publications on clinical research. Four important categories in clinical research can be distinguished (see Table 1), on which we will emphasize in this study (3).
Table 1. Publication topic.
Besides original publications, systematic reviews are also important for decision making in patient care. They collect and summarize all existing publications and are considered the highest quality, i.e., level 1a, evidence (4).
In the past, similar studies have been performed, showing a constant amount of 77% clinical research in'69,'79, and'89 in four major Otolaryngology journals (5). In 1999, clinical research accounted for 72% of publications in four major journals (6). Six major Otolaryngology journals were reviewed for the years 1993 and 2003, showing an increase in clinical research from 72 to 73% (7).
The purpose of our study is to provide insight in the volume and type of clinical research that is published in 1 year, in the field of Otorhinolaryngology. In addition, we compare leading Otolaryngology specialty journals to journals outside this field, based on their impact factor (8).
Materials and Methods
Selection of Journals
Data collection was carried out in February 2012. We identified leading journals by their impact factor. We selected the top 10 impact factor Otolaryngology journals, using the 2011 impact factor (8). We then searched for medical journals outside the field of Otolaryngology. We used the 2011 impact factor to rank journals from high to low and selected the first 20 journals that were likely to publish articles related to Otolaryngology. The journals were selected based on their scope. The in- and exclusion criteria can be found in Figure 1. The overview of all reviewed journals can be found online (Appendix 1). Two authors independently selected journals; initial disagreement was resolved by consensus (NK and GR).
Figure 1. Selection of journals.
Selection of Publications
We retrieved full texts of all citable articles, i.e., peer-reviewed articles from the selected Otolaryngology journals. From the selected medical journals two authors independently retrieved full texts of all citable articles concerning Otolaryngology based on title or abstract, initial disagreement was resolved by consensus (NK and GR).
We selected all original publications on health outcomes in patients, with a determinant or outcome considered relevant for patient care (3). (For selection criteria see Table 2) We also selected systematic reviews and meta-analyses that were based on original publications, relevant for patient care. Case reports were excluded (4).
Table 2. Publication types.
The selected studies were further classified based on the purpose of their research question, e.g., therapy, diagnosis, prognosis or etiology (see Table 1 for definitions) (3).
Two authors (NK and GR) independently retrieved and reviewed all publications. Initial disagreement on selection and categorization of articles was discussed with a third author (GvdH) until agreement was reached; the selection is therefore based on a full consensus.
Results
Journals
The 10 selected Otolaryngology journals can be found in Table 3 (impact factor 1.8 to 2.8). The scope of these journals can be found online (Appendix 2), two did not publish clinical evidence. The 20 selected medical journals had an impact factor varying from 6.0 to 101.8 (Table 3). A complete list of all evaluated medical journals is available online (Appendix 1). The selection process is shown in Figure 1. The titles of the selected journals can be found in Table 3.1 (Otolaryngology journals) and Table 3.2 (medical journals).
Table 3.1. Results 1. Type of research in Otolaryngology journals.
Table 3.2. Results 2. Type of research in selected medical journals.
Publications
From 1,500 articles in Otolaryngology journals, we identified 694 (46%) original publications on clinical research and 27 (2%) systematic reviews (Figure 2). Of 5,462 citable articles in selected medical journals, 122 (2%) were related to Otolaryngology. Of these, 102 (83%) were original publications and 2 (0,04%) were systematic reviews. The different research questions are shown in Figure 2. Most publications concern therapy, followed by prognosis and etiology, least represented is diagnosis. The proportion of publications on prognosis and etiology research is lower in Otolaryngology journals. For diagnosis and therapy there are no differences. The results per journal can be found in Tables 3.1,3.2. The variation in the proportion of clinical research between journals could be explained by the different scopes of the journals; JARO only publishes basic research and Current Opinion in Otolaryngology only publishes (non-systematic) reviews (see online Appendix 2). If we exclude these journals from the analysis, the proportion of clinical research increases to 51%.
Figure 2. Selection of publications.
Discussion
Synopsis of Key Findings
We set out to find the amount of clinical research in Otolaryngology, published both in leading Otolaryngology journals and in medical journals outside this field.
Two percent of Otolaryngology related research is published in journals outside the field of Otolaryngology. In selected Otolaryngology journals, 46% (95% confidence interval 44; 49) of publications relates to clinical research. The proportion increased to 51% (95% confidence interval 48; 53) after excluding two journals that do not publish clinical research.
We found that the impact factor of medical journals outside the field of Otolaryngology (6.0 to 101.8) is higher than the impact factor of Otolaryngology journals (1.8 to 2.8).
Regarding research questions, 40% of publications was related to therapy.
Comparison to Previous Studies
The proportion of clinical research in Otolaryngology is lower than the 75% reported previously (5–7) The difference may be explained by selection of different journals (the only identical journal included in the previous studies was the Laryngoscope), by our use of possible more stringent definitions than in previous research. It could also indicate an actual decrease in clinical evidence. Compared to other specialties, Otolaryngology journals achieve similar or higher rates of clinical evidence. For example, previous studies found an amount of clinical evidence in Urology journals of 35% (2002–2010) (9), 53% in anesthetics (2000–2009), (10) 11% in plastic surgery (2002) (11) and 24% in ophthalmology (2002) (12).
Limitations of Our Study
We used the impact factor of 2011, which relates to publications from 2010 and 2011, but for this study we only selected publications from 2010. We selected 30 medical journals and reviewed over 300 journals. These journals were selected retrospectively, therefore selection bias could have occurred. The selected publications can be judged outdated. However, it takes some time for studies to become available full text and to be indexed. Then it also takes several years before studies are adopted in daily practice (13). When we look at clinical practice guidelines, it is common to find references of studies of 2010 and before. Therefore, our results are still important and informative. We selected articles based on the impact factor of the journal they were published in, since we wanted to show results for leading journals, since they are often read and looked to for relevant research. With this selection approach, we systematically evaluated almost 12,500 articles. However, we might have missed publications with our search strategy, so the actual amount of clinical outcome research could be either lower or higher.
Implications for Clinicians and Researchers
For Otolaryngology journals, it is striking that some high impact factor journals do not strive to publish clinical research. In addition, our results show that 2% of Otolaryngology related clinical outcome research is published in journals outside the field of Otolaryngology. These findings supports results from different specialties, i.e., that important clinical studies are often not published in specialty journals (14). Moreover, Otolaryngology related research of substantial quality might be published in medical journals outside the field of Otolaryngology, since higher impact factors can be achieved. Doctors should therefore look beyond their specialized journals when searching for evidence. Forty percent of publications we found, report about therapy, which is a similar result to previous studies (7). Studies concerning etiology, prognosis and diagnosis are less common. This implies that the emphasis of researchers and journals is more on therapy than diagnosis, prognosis or etiology. Yet they should realize that these purpose categories are also important for clinical practice (3). Our results show a limited number of systematic reviews (both in and outside of Otolaryngology journals). Systematic reviews are of high importance since they sum op the results of existing studies (4). Therefore, we highly recommend that the amount of systematic reviews should increase, both in and outside of Otolaryngology journals.
For evidence-based practice, clinical original studies are also of vital importance (1, 2). We must therefore also ensure an increase of this type of research, to improve and optimize medical practice (15). This applies particularly to Otolaryngology journals, journals outside the field of Otolaryngology showed a better balance between clinical and non-clinical (e.g. fundamental) research (50 vs. 83%).
The amount of clinical evidence is predominantly determined by the scope of a journal and the choice of the editors and reviewers, but also by amount of studies that are conducted and submitted (15). On one hand, this implies that editors and reviewers of journals should watch for balance between publication of clinical and fundamental research. On the other hand, researchers and doctors involved in research, are encouraged to publish clinical evidence (15).
Author Contributions
NK conception and design, acquisition of data, analysis and interpretation of data, drafting and revising the article, final approval of the version to be submitted and any revised version. GR acquisition of data, analysis and interpretation of data, drafting the article, final approval of the version to be submitted and any revised version. MA analysis and interpretation of data, revising the article, final approval of the version to be submitted and any revised version. GvdH conception and design, analysis and interpretation of data, drafting and revising the article, final approval of the version to be submitted and any revised version.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsurg.2019.00018/full#supplementary-material
References
1. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. (2017) 390:415–23. doi: 10.1016/S0140-6736(16)31592-6
2. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. (2014) 383:156–65. doi: 10.1016/S0140-6736(13)62229-1
3. Grobbee DE, Hoes AW. Clinical Epidemiology: Principles, Methods, and Applications for Clinical Research. London: Jones and Bartlett Publishers (2009)
4. http://www.cebm.net/levels_of_evidence.asp (accessed: August 11, 2016)
5. Rosenfeld R. Clinical research in otolaryngology journals. Arch Otolaryngol Head Neck Surg. (1991) 117:164–70.
6. Bentsianov B, Boruk M, Rosenfeld RM. Evidence-based medicine in otolaryngology journals. Otolaryngol Head Neck Surg. (2002) 126:371–6.
7. Wasserman JM, Wynn R, Bash TS, Rosenfeld RM. Levels of evidence in otolaryngology journals. Otolaryngol Head Neck Surg. (2006) 134:717–9. doi: 10.1016/j.otohns.2005.11.049
9. Mani J, Makarević J, Juengel E, Ackermann H, Nelson K, Haferkamp A, et al. Publication of original research in urologic journals - a neglected orphan? PLoS ONE. (2012) 7:e52420. doi: 10.1371/journal.pone.0052420
10. Li Z, Qiu LX, Wu FX, Yang LQ, Sun S, Yu WF. Scientific publications in international anaesthesiology journals: a 10-year survey. Anaesth Intensive Care. (2011) 39:268–73. doi: 10.1007/s00540-010-1073-4
11. Huemer GM, Bauer T, Gurunluoglu R, Sakho C, Oehlbauer M, Dunst KM. Analysis of publications in three plastic surgery journals for the year 2002. Plast Reconstr Surg. (2004) 114:1147–54. doi: 10.1097/01.PRS.0000135851.23100.7E
12. Lai TY, Leung GM, Wong VW, Lam RF, Cheng AC, Lam DS. How evidence-based are publications in clinical ophthalmic journals? Invest Ophthalmol Vis Sci. (2006) 47:1831–8. doi: 10.1167/iovs.05-0915
13. Glasziou P, Haynes B. The paths from research to improved health outcomes (editorial). ACP J Club. (2005) 142:8–10. doi: 10.7326/ACPJC-2005-142-2-A08
14. McKibbon KA, Wilczynski1 NL, Haynes R. What do evidence-based secondary journals tell us about the publication of clinically important articles in primary healthcare journals? BMC Med. (2004) 2:33. doi: 10.1186/1741-7015-2-33
Keywords: evidence-based medicine, evidence-based practice, impact factor, otolaryngology, therapy, diagnosis, prognosis, etiology
Citation: Kaper NM, Ramakers GGJ, Aarts MCJ and van der Heijden GJMG (2019) Publications on Clinical Research in Otolaryngology–A Systematic Analysis of Leading Journals in 2010. Front. Surg. 6:18. doi: 10.3389/fsurg.2019.00018
Received: 25 January 2019; Accepted: 18 March 2019;
Published: 09 April 2019.
Edited by:
Pavel Dulguerov, Geneva University Hospitals (HUG), SwitzerlandReviewed by:
Maria Grosheva, Universität zu Köln, GermanyA. B. Zulkiflee, University Malaya Medical Centre, Malaysia
Copyright © 2019 Kaper, Ramakers, Aarts and van der Heijden. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Nina M. Kaper, kno@umcutrecht.nl
orcid.org/0000-0002-8849-8645