The case for development of a core outcome set (COS) and supplemental reporting guidelines for influenza vaccine challenge trial research in swine

Previously, we systematically reviewed more than 20 years of influenza vaccine challenge trial research in pigs to answer the question, “does vaccinating sows protect offspring?” Overall, most studies were well designed but clinical heterogeneity made between-study comparisons challenging. Studies varied by samples, outcomes, and assays selected for measurement. Additionally, data essential for inclusion of findings in meta-analyses were often insufficiently reported and as a result, summary effect measures were either not derived or were not meaningful. Clinical heterogeneity and reporting issues complicate and limit what can be learned cumulatively from research and both represent two types of avoidable research waste. Here, we illustrate each concern using data collected tangentially during the systematic review and propose two corrective strategies, both of which have broad applicability across veterinary intervention research; (i) develop a Core Outcome Set (COS) to reduce unnecessary clinical heterogeneity in future research and (ii) encourage funders and journal editors to require submitted research protocols and manuscripts adhere to established reporting guidelines. As a reporting corollary, we developed a supplemental checklist specific to influenza vaccine challenge trial research in swine and propose that it is completed by researchers and included with all study protocol and manuscript submissions. The checklist serves two purposes: as a reminder of details essential to report for inclusion of findings in meta-analyses and sub-group meta-analyses (e.g., antigenic or genomic descriptions of influenza vaccine and challenge viruses), and as an aid to help synthesis researchers fully characterize and comprehensively include studies in reviews.

1 Introduction—avoidable research waste and corrective strategies

In 2014, evidence-based medicine (EBM) methodologists authored a series of seminal articles in the journal, The Lancet, describing five areas of avoidable waste in biomedical research and offered corrective strategies for each (1–5). Multiple stakeholders use research for decision making but recognizing outcomes most relevant to users were not consistently selected, measured, or reported in research, methodologists also focused on standardization of endpoints (6, 7). Products of this effort include:

2010 – development of the COSMIN checklist (Consensus-based Standards for the selection of health status Measurement INstruments) as an aid for evaluating the methodological quality of studies investigating measurement properties (8, 9).

• 2014 – formation of COMET, the Core Outcome Measures in Effectiveness Trials initiative (9).

• 2016 – establishing COS-STAR, Core Outcome Set-STAndards for Reporting as reporting guidelines for COS development studies (10, 11).

• 2017 – establishing COS-STAD, Core Outcome Set-STAndards for Development to outline minimum standards for the design of COS development studies (12).

• 2019 – providing COS-STAP, Core Outcome Set-STAndardized Protocol Items, a checklist of 13 items considered essential documentation in protocols for COS development studies (13).

Core outcome sets (COS) are adopted to harmonize inclusion of essential elements of study design in clinical research (2, 11). A COS is “an agreed minimum set of outcomes that should be measured and reported in all clinical trials of a specific disease or trial population”; it is disease or population specific but it is not trial specific, meaning it is a recommendation of what should be minimally measured and reported in all clinical trials, but not how (9, 14, 15). Establishment of core outcome sets (COS) address issues of research waste associated with avoidable differences from one study to the next in the elements of study design (i.e., clinical heterogeneity) (2, 12).

Adoption of a COS approach reduces waste and improves the value of research to users in three ways; (1) it ensures outcomes most relevant to stakeholders are included in research, (2) it reduces outcome reporting bias (i.e., prevents selective reporting of only a sub-set of measured outcomes), and (3) it ensures all trials contribute usable information in meta-analyses (14). Inclusion of each study’s data in meta-analyses means that resources invested in primary research may be further leveraged through synthesis (16–18). Hundreds of core outcome sets (COS) have been developed in human healthcare (19) but we are aware of only two COS initiatives in veterinary care; COSCAD’18 for atopic dermatitis in dogs (20) and development of a COS for feline chronic kidney disease treatment trials (21). However, awareness is growing and recently, Sargeant et al. (22) described how a COS approach maximizes the utility of intervention research trials conducted in swine populations.

Within the context of The Lancet series (2, 5), we revisit our systematic review and meta-analysis of 20 years of influenza challenge trial research in swine (published between 1990 and May 2021) (23) to explore and illustrate two types of avoidable waste found in that body of evidence. Our perspective for this work was gained through prior investigation of the barriers that slow translation of influenza research into useful knowledge for swine practitioners (23–25). We use data collected tangentially during the systematic review to identify clinical heterogeneity and reporting insufficiencies and based on this, propose two corrective strategies; development of a COS using standard and established Delphi consensus building processes (12), and, in addition to promoting use of established reporting guidelines, propose adoption of a novel supplemental reporting checklist specific to influenza vaccine challenge trial research in swine.

2 Clinical heterogeneity and reporting insufficiency in swine MDI challenge trial research

The systematic review was conducted to answer the question of whether the common industry practice of vaccinating sows against influenza conferred protection (via maternally derived immunity (MDI)) to their offspring (23). Data on six different outcomes were characterized and extracted for meta-analyses. Outcomes included three direct measures of infection, a gold standard immune correlate of protection (CoP), and two clinical signs. Measures of effect (also known as treatment measures or effect sizes) were impacted by the match between vaccine antigens and corresponding challenge virus(es), but overall, challenge trial evidence neither supported nor refuted vaccination of sows to protect piglets (23).

Post hoc, we found the strength of the systematic review and specifically, confidence in summary effect measures, was reduced due clinical heterogeneity, and due to incomplete reporting of trial information and data. Reporting was not necessarily incomplete from the standpoint of communicating finding of individual studies, but rather, details were frequently omitted that were essential for inclusion in meta-analyses (i.e., measures of centrality, measures of dispersion, group sizes) and for inclusion in sub-group meta-analyses (i.e., antigenic characterization of vaccine and challenge viruses) (23). Table 1 is a summary of the number of studies that included one or more of the six outcomes versus the number of studies that were included in the meta-analysis (MA) of each outcome. Proportionately, few eligible studies were included in meta-analyses.

Table 1

Table 1. Number of studies by reported outcomes versus studies reporting sufficient data for inclusion in meta-analysis (N = 15^a influenza challenge studies in piglets with vaccine-derived MDI published between 1990 and 2021, and reporting at least 1 of 6 eligible outcomes).

2.1 Clinical heterogeneity and elements of study design

To illustrate clinical heterogeneity we looked at three elements of study design; (i) outcomes measured, (ii) samples collected, and (iii) assays employed, then further characterized elements into sub-elements and identified use of each across all studies. Overall, the body of research was complex applying different combinations of samples collected, assays used, and outcomes measured, both within and across studies.

2.2 Clinical heterogeneity and network analysis

We used methods of network analyses to generate displays showing the scope and relatedness of each of the three elements as applied across all studies (N = 16). Figure 1 and Supplementary Figures S1, S2, display the number of studies jointly including the same sub-elements of outcomes measured, samples collected, and assays used across all studies. The Delphi method for reaching consensus is a preferred approach for developing a COS and this type of visual can, at the onset of the process, help Delphi study participants to understand the extent of existing clinical heterogeneity (9, 12, 21, 26).

Figure 1

Figure 1. Network analysis of outcomes measured across 16 influenza challenge trials in piglets with vaccine-derived maternal immunity. Data was collected tangentially during the systematic review of IAV-S influenza challenge trials in piglets with maternally derived immunity (MDI) from vaccinated sows (N = 16) (23); Red circles (nodes), outcome measured; numbers assigned each grey line (edge), number of studies jointly reporting both outcomes (thickness of line increases with increased frequency of joint reporting); CMI, all non-immunoglobulin immune responses including cell mediated immunity; vaerd, vaccine associated enhanced respiratory disease; Virus_quant, virus titres extrapolated from PCR measurements or from virus isolation methods; lung_path, observed macro or microscopic evaluation of lung lesions; transmissn, virus transmission from an infected pig to an uninfected pig; virus_detect, virus detection via RT-PCR or virus isolation; clin_sign, clinical signs; antibody, immunoglobulin titres; other, microtracheal lesions.

2.2.1 Network analysis of outcomes measured

Outcomes were categorized into 9 sub-elements. Figure 1 shows the connectivity of each across all 16 studies. Nodes (red circles) represent the sub-element, and the edges (connecting lines) equal the number of studies jointly measuring both sub-elements. Virus detection and antibody titres were measured in all studies; each was most frequently paired with clinical signs (n = 12) and virus quantification (n = 12). Other frequent pairings included lung pathology with each of virus detection (n = 9), virus quantification (n = 9), and antibody measures (n = 9). Virus quantification and clinical signs were jointly measured across more than half of all studies (n = 9).

2.2.2 Network analysis of samples collected

Samples were categorized into 17 sub-elements (see Supplementary Figure S1). Multiple sub-elements were collected in each study and the most frequent pairing of samples across studies was nasal swabs with serum (n = 14), both of which were also paired frequently with dyspnea (n = 10). Next most frequent pairings were fever with each of serum (n = 12), nasal swabs (n = 11), or dyspnea (n = 10). Half of all studies (n = 8) included both collection of serum and bronchioalveolar lavage fluids (collected post mortem).

2.2.3 Network analysis of assays used

Assays were categorized into 15 sub-elements (see Supplementary Figure S2); most frequent pairings were ELISA assays (ELISA_2) with cell culture (for virus isolation) (n = 13), followed by ELISAs with measure of hemagglutination inhibition titres (HAI) (n = 12), and HAI with cell culture (n = 10). Half of all studies included measures of macroscopic lung lesions (L_macro) with each of lung histological lesions (L_micro) (n = 8), HAI (n = 9), immunoglobulins (ELISA_2) (n = 8), or cell culture (n = 8). All microscopic reporting of lung lesions (L_micro) was paired with (HAI) (n = 8).

2.3 Summary of the clinical elements of study design across all studies (N = 16)

All studies included as outcomes measures of immunoglobulin responses and of virus detection; inclusion of other outcomes was variable (see Table 1). Samples collected and assays employed to measure each of the six outcomes varied across all 16 studies and are summarized in Supplementary Figures S3, S4, respectively.

Nasal swabs were used to collect samples for virus detection in 14/16 studies (Supplementary Figure S3). Three approaches were taken to demonstrate direct evidence of virus in piglets post challenge (Supplementary Figure S4); (1) culture of live virus (n = 13), (2) detection of viral RNA nucleic acids using polymerase chain reaction (PCR) methods (n = 6), and (3) histological identification in situ of viral antigen in sectioned tissues [immunohistochemistry (IHC)] (n = 5). Virus was quantified using cell culture methods (n = 10) and RT-qPCR methods (n = 4).

Serum immunoglobulin titres were measured in all but 1 study using enzyme linked immunosorbent assays (ELISA) (n = 15) (Supplementary Figure S4). ELISAs also varied with respect to the type of immunoglobulins measured; as IgG and /or IgA isotypes or as immunoglobulins specifically against conserved viral proteins NP (nucleoprotein) or M (matrix protein). The HAI assay, which is the gold standard correlate of protection, was used in most studies (n = 13) to identify sub-type and strain specific immunologic responses to influenza A viruses of swine.

Although few studies included outcomes measuring cell mediated immune responses (CMI), the assays used were varied; multiparameter flow cytometry (n = 6) was used to assess T-cell proliferation (staining for detection of CD4+, CD8+, CD3+, γδ TCR+ peripheral blood mononuclear (PBMN) cell populations) and T-cell priming (staining for CD25, IFN-gamma, and IL-10) (Supplementary Figure S4). ELISA based assays including the enzyme-linked immunospot (ELISpot) assay (n = 1), used for detection of IFN-gamma secreting cells, and a multiplex ELISA (n = 2) used to detect cytokine production. Clinical signs were measured as a stand-alone measure or as part of a composite score in 12 of 16 studies and are summarized in Supplementary Figure S5. Fever was the most consistently measured and reported clinical sign.

2.4 Supplemental reporting of influenza vaccine challenge trials in swine

Incomplete reporting of study methods and results hinders assessment for internal and external validity (27, 28), and in veterinary medicine is a cause of frequent exclusion of primary research studies from systematic reviews and meta-analyses (18, 29–34). In our review, studies were excluded from sub-group meta-analyses if hemagglutinin and neuraminidase antigens in vaccines were poorly characterized (23). This further diminished the value of inferences derived through synthesis. Therefore, we encourage funders and journal editors require research protocols and manuscripts adhere to established reporting guidelines such as REFLECT (available on the Meridian Network at https://meridian-network.org/).

Additionally, we developed a reporting checklist specific to influenza vaccine challenge trails in swine (see Supplementary Tables S1, S2) and encourage researchers to complete and include the checklist as a supplemental document with all manuscript submissions. The checklist itemizes sub-elements of study design important for overall contextual understanding of influenza vaccine research (i.e., outcomes, samples and assays). The intent is that it serves as a reminder to researchers of essential items to report (35), and as an aid for synthesis researchers when identifying studies and extracting data for inclusion in systematic reviews and meta-analyses (31, 36). Consistent reporting of outcome, assay and sample sub-elements may also help to improve researcher, reviewer, and reader awareness of the full extent of avoidable clinical heterogeneity (8, 9).

3 Discussion

3.1 Objectives, endpoints, and definitions of influenza vaccine protection

Vaccine protection is a non-specific term and context is important for interpretation; protection from infection must be distinguished from protection against an undesired clinical endpoint (37) and no single outcome conveys protection against infection or disease (38). From a societal perspective, desired endpoints are conditioned on stakeholders’ need to inform decisions on implementation of vaccines in the field. For researchers involved in vaccine development work, desired endpoints are a function of the phase of research for which they are engaged (39, 40). Examples of researcher objectives, societal perspectives, and corresponding vaccine research endpoints and measures of protection against in influenza A viruses in swine (IAV-S) are listed in Supplementary Table S3.

In human influenza vaccine research, researchers’ objectives differ by each distinct phase of research (see Supplementary Table S3; Supplementary Figure S6) (38, 41–43). Phase “0″ is the proof-of-concept phase for candidate vaccines. In Phase I to III clinical trials, convenient methods and assays to measure efficacy are validated, safety, dosages, and schedules are established, and approval for market use is established. In Phase IV clinical trials, post marketing evaluation of field effectiveness is established. Compared to human vaccines, approval of veterinary vaccines is expedited due to:

i. Fewer restrictions governing conduct of same species challenge trials (44–46).

ii. Veterinary vaccines are commonly approved based on Phase II challenge trial evidence (44).

iii. Post-marketing evaluation of vaccine performance is not required (44).

iv. A regulatory framework exists for restricted market use of non-licensed autogenous vaccines without proof of efficacy (46).

Desired endpoints overlap in early phase and late phases of vaccine research (Supplementary Table S3) (38). Validating correlates of protection (CoP), specifically immune CoPs, is often done using challenge trials in animal models, where same-species challenge models are optimal (47–49). Historically, hemagglutination inhibition (HAI) serologic assays have been the gold standard CoP for evaluating efficacy of inactivated vaccines against influenza A virus (IAV) (50), however because understanding of immune responses to influenza is incomplete, multiple outcomes must be assessed to accurately predict vaccine performance (37, 51). Additionally, use of live vaccines and newer vaccine technologies means both cell-mediated CoPs and direct measures of virus infection are needed to assess protection (50, 52–54).

There are gaps in understanding of how a host’s immune responses are coordinated to eliminate virus following primary infections, and of the impact of prior or original virus exposure on immunologic responses to secondary strain-homologous and to strain-heterologous IAV exposures (55). Universal vaccines and novel vaccine platforms typically elicit immunologic responses to non-dominate epitopes (i.e., the dominant HA receptor binding sites are not targeted) and as emphasis shifts to the design of such vaccines, so too will study of unintended adverse outcomes such as vaccine enhanced acute respiratory disease (VEARD) (56–61).

3.2 The role of challenge trials and swine influenza research in vaccine development

Although rare in human medicine (45, 62), challenge trials were the most frequently reported study design in IAV-S vaccine research in swine published since 1990 (25). For each of the 16 MDI challenge studies, the author’s stated objective(s) and study endpoints are summarized in Supplementary Table S4, indicating also where objectives correspond to one or more of the five critical R&D immunologic issues outlined in The Influenza Vaccine Research and Development (R&D) Roadmap (51).

Within a One Health context., IAV-S vaccine challenge research in swine contributes to the larger influenza research community of practice, and in this light, development of a COS for IAV-S research in swine (using a Delphi process) may serve as a template for development of COS for influenza vaccine research in other animal species (63, 64). Similarly, adoption of the proposed checklist may also serve as a multi-species template for consistent reporting of essential elements important for research synthesis and ultimately for the interpretation of vaccine challenge trial studies in animals.

4 Conclusion

Within the context of avoidable research waste, we illustrated how clinical heterogeneity and reporting insufficiencies led to substantive exclusion of IAV-S MDI challenge trials in swine from subsequent qualitative and quantitative synthesis of the body of research evidence. We advanced two corrective actions; development of a core outcome set (COS) using Delphi consensus building methods, and in addition to encouraging adherence to established reporting guidelines, use of a novel supplemental reporting checklist specific to influenza challenge trials in swine. We suggested a completed checklist accompanies all primary research manuscripts as an aid to funders, editorial reviewers, readers, and synthesis researchers, for improving contextual understanding, and to facilitate charting and extracting IAV-S specific details during synthesis.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

SK: Conceptualization, Methodology, Project administration, Investigation, Data curation, Formal analysis, Funding acquisition, Visualization, Writing – original draft, Writing – review & editing. FA: Conceptualization, Methodology, Investigation, Writing – review & editing. AOC: Conceptualization, Writing – review & editing, RF: Writing – review & editing. TOC: Writing – review & editing. ZP: Conceptualization, Methodology, Investigation, Data curation, Formal analysis, Software, Funding acquisition, Visualization, Resources, Supervision, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was funded by an Ontario Veterinary College Fellowship grant (SK) https://ovc.uoguelph.ca/ and the Ontario Ministry of Food Agriculture and Rural Affairs (OMAFRA) https://www.ontario.ca/page/agricultural-research-and-innovation, project# KTT2015-10253 (ZP). The Ontario Veterinary College and OMAFRA had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Acknowledgments

We thank the Ontario Veterinary College and the Ontario Ministry of Agriculture Food and Rural Affairs (OMAFRA) for funding this work, and Drs. Marnie Brennan and Lisa Waddell for their advice during conceptualization, and Dr. Jan Sargeant for her continued conceptual, technical and editorial guidance throughout all aspects of the larger project.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fvets.2025.1465926/full#supplementary-material

References

1. Chalmers, I, Bracken, MB, Djulbegovic, B, Garattini, S, Grant, J, Gülmezoglu, AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. (2014) 383:156–65. doi: 10.1016/S0140-6736(13)62229-1

PubMed Abstract | Crossref Full Text | Google Scholar

2. Ioannidis, JPA, Greenland, S, Hlatky, MA, Khoury, MJ, Macleod, MR, Moher, D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. (2014) 383:166–75. doi: 10.1016/S0140-6736(13)62227-8

PubMed Abstract | Crossref Full Text | Google Scholar

3. Al-Shahi Salman, R, Beller, E, Kagan, J, Hemminki, E, Phillips, RS, Savulescu, J, et al. Increasing value and reducing waste in biomedical research regulation and management. Lancet. (2014) 383:176–85. doi: 10.1016/S0140-6736(13)62297-7

PubMed Abstract | Crossref Full Text | Google Scholar

4. Chan, AW, Song, F, Vickers, A, Jefferson, T, Dickersin, K, Gøtzsche, PC, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. (2014) 383:257–66. doi: 10.1016/S0140-6736(13)62296-5

PubMed Abstract | Crossref Full Text | Google Scholar

5. Glasziou, P, Altman, DG, Bossuyt, P, Boutron, I, Clarke, M, Julious, S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. (2014) 383:267–76. doi: 10.1016/S0140-6736(13)62228-X

PubMed Abstract | Crossref Full Text | Google Scholar

6. Tugwell, P, Boers, M, Brooks, P, Simon, L, Strand, V, and Idzerda, L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. (2007) 8:38. doi: 10.1186/1745-6215-8-38

PubMed Abstract | Crossref Full Text | Google Scholar

7. Clarke, M. Standardising outcomes for clinical trials and systematic reviews. Trials. (2007) 8:1–3. doi: 10.1186/1745-6215-8-39

PubMed Abstract | Crossref Full Text | Google Scholar

8. Mokkink, LB, Terwee, CB, Knol, DL, Stratford, PW, Alonso, J, Patrick, DL, et al. The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: a clarification of its content. BMC Med Res Methodol. (2010) 10:22. doi: 10.1186/1471-2288-10-22

PubMed Abstract | Crossref Full Text | Google Scholar

9. Prinsen, CAC, Vohra, S, Rose, MR, King-Jones, S, Ishaque, S, Bhaloo, Z, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a “core outcome set”. Trials. (2014) 15:1–7. doi: 10.1186/1745-6215-15-247

PubMed Abstract | Crossref Full Text | Google Scholar

10. Kirkham, JJ, Gorst, S, Altman, DG, Blazeby, J, Clarke, M, Devane, D, et al. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials. (2015) 16:373. doi: 10.1186/s13063-015-0913-9

PubMed Abstract | Crossref Full Text | Google Scholar

11. Kirkham, JJ, Gorst, S, Altman, DG, Blazeby, JM, Clarke, M, Devane, D, et al. Core Outcome Set–STAndards for Reporting: the COS-STAR statement. PLoS Med. (2016) 13:1–11. doi: 10.1371/journal.pmed.1002148

PubMed Abstract | Crossref Full Text | Google Scholar

12. Kirkham, JJ, Davis, K, Altman, DG, Blazeby, JM, Clarke, M, Tunis, S, et al. Core Outcome Set-STAndards for Development: the COS-STAD recommendations. PLoS Med. (2017) 14:1–10. doi: 10.1371/journal.pmed.1002447

PubMed Abstract | Crossref Full Text | Google Scholar

13. Kirkham, JJ, Gorst, S, Altman, DG, Blazeby, JM, Clarke, M, Tunis, S, et al. Core Outcome Set-STAndardised Protocol items: the COS-STAP statement. Trials. (2019) 20:1–7. doi: 10.1186/s13063-019-3230-x

PubMed Abstract | Crossref Full Text | Google Scholar

14. Williamson, P, Altman, D, Blazeby, J, Clarke, M, and Gargon, E. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy. (2012) 17:1–2. doi: 10.1258/jhsrp.2011.011131

PubMed Abstract | Crossref Full Text | Google Scholar

15. Williamson, PR, Altman, DG, Blazeby, JM, Clarke, M, Devane, D, Gargon, E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. (2012) 13:132. doi: 10.1186/1745-6215-13-132

PubMed Abstract | Crossref Full Text | Google Scholar

16. McCrackin, MA, Helke, KL, Galloway, AM, Poole, AZ, Salgado, CD, Marriott, B, et al. Effect of antimicrobial use in agricultural animals on drug-resistant foodborne campylobacteriosis in humans: a systematic literature review. Crit Rev Food Sci Nutr. (2016) 56:2115–32. doi: 10.1080/10408398.2015.1119798

PubMed Abstract | Crossref Full Text | Google Scholar

17. Burns, MJ, and O’Connor, AM. Assessment of methodological quality and sources of variation in the magnitude of vaccine efficacy: a systematic review of studies from 1960 to 2005 reporting immunization with Moraxella bovis vaccines in young cattle. Vaccine. (2008) 26:144–52. doi: 10.1016/j.vaccine.2007.10.014

PubMed Abstract | Crossref Full Text | Google Scholar

18. Wilhelm, B, Rajić, A, Parker, S, Waddell, L, Sanchez, J, Fazil, A, et al. Assessment of the efficacy and quality of evidence for five on-farm interventions for Salmonella reduction in grow-finish swine: a systematic review and meta-analysis. Prev Vet Med. (2012) 107:1–20. doi: 10.1016/j.prevetmed.2012.07.011

PubMed Abstract | Crossref Full Text | Google Scholar

19. Dodd, S, Clarke, M, Becker, L, Mavergames, C, Fish, R, and Williamson, PR. A taxonomy has been developed for outcomes in medical research to help improve knowledge discovery. J Clin Epidemiol. (2018) 96:84–92. doi: 10.1016/j.jclinepi.2017.12.020

PubMed Abstract | Crossref Full Text | Google Scholar

20. Olivry, T, Bensignor, E, Favrot, C, Griffin, CE, Hill, PB, Mueller, RS, et al. Development of a core outcome set for therapeutic clinical trials enrolling dogs with atopic dermatitis (COSCAD’18). BMC Vet Res. (2018) 14:4–11. doi: 10.1186/s12917-018-1569-y

PubMed Abstract | Crossref Full Text | Google Scholar

21. Doit, H, Dean, RS, Duz, M, Finch, NC, and Brennan, ML. What outcomes should be measured in feline chronic kidney disease treatment trials? Establishing a core outcome set for research. Prev Vet Med. (2021) 192:105348. doi: 10.1016/j.prevetmed.2021.105348

PubMed Abstract | Crossref Full Text | Google Scholar

22. Sargeant, JM, O’Connor, AM, O’Sullivan, TL, and Ramirez, A. Maximizing value and minimizing waste in clinical trials in swine: selecting outcomes to build an evidence base. J Swine Health Prod. (2023) 31:29–35. doi: 10.54846/jshap/1300

Crossref Full Text | Google Scholar

23. Keay, S, Poljak, Z, Alberts, F, O’Connor, A, Friendship, R, O’Sullivan, TL, et al. Does vaccine-induced maternally-derived immunity protect swine offspring against influenza a viruses? A systematic review and meta-analysis of challenge trials from 1990 to may 2021. Animals. (2023) 13:3085. doi: 10.3390/ani13193085

PubMed Abstract | Crossref Full Text | Google Scholar

24. Keay, S, Sargeant, JM, O’Connor, A, Friendship, R, O’Sullivan, T, and Poljak, Z. Veterinarian barriers to knowledge translation (KT) within the context of swine infectious disease research: an international survey of swine veterinarians. BMC Vet Res. (2020) 16:1–17. doi: 10.1186/s12917-020-02617-8

PubMed Abstract | Crossref Full Text | Google Scholar

25. Keay, S, Poljak, Z, Klapwyk, M, O’Connor, A, Friendship, RM, O’Sullivan, TL, et al. Influenza a virus vaccine research conducted in swine from 1990 to may 2018: a scoping review. PLoS One. (2020) 15:e0236062. doi: 10.1371/journal.pone.0236062

PubMed Abstract | Crossref Full Text | Google Scholar

26. McMenamin, ME, Bond, H, Sullivan, S, and Cowling, B. Estimation of relative vaccine effectiveness in influenza. Epidemiology. (2022) 33:334–45. doi: 10.1097/EDE.0000000000001473

Crossref Full Text | Google Scholar

27. Sargeant, JM, and O’Connor, AM. Conducting systematic reviews of intervention questions II: relevance screening, data extraction, assessing risk of bias, presenting the results and interpreting the findings. Zoonoses Public Health. (2014) 61:39–51. doi: 10.1111/zph.12124

PubMed Abstract | Crossref Full Text | Google Scholar

28. O’Connor, AM, Sargeant, JM, Gardner, IA, Dickson, JS, Torrence, ME, Dewey, CE, et al. The REFLECT statement: methods and processes of creating reporting guidelines for randomized controlled trials for livestock and food safety. Zoonoses Public Health. (2010) 57:95–104. doi: 10.1111/j.1863-2378.2009.01311.x

PubMed Abstract | Crossref Full Text | Google Scholar

29. Lean, IJ, Lucy, MC, McNamara, JP, Bradford, BJ, Block, E, Thomson, JM, et al. Invited review: recommendations for reporting intervention studies on reproductive performance in dairy cattle: improving design, analysis, and interpretation of research on reproduction. J Dairy Sci. (2016) 99:1–17. doi: 10.3168/jds.2015-9445

PubMed Abstract | Crossref Full Text | Google Scholar

30. Winder, CB, Churchill, KJ, Sargeant, JM, LeBlanc, SJ, O’Connor, AM, and Renaud, DL. Invited review: completeness of reporting of experiments: REFLECTing on a year of animal trials in the journal of dairy science. J Dairy Sci. (2019) 102:4759–71. doi: 10.3168/jds.2018-15797

PubMed Abstract | Crossref Full Text | Google Scholar

31. Moura, CAA, Totton, SC, Sargeant, JM, Sullivan, TLO, and Linhares, DCL. Evidence of improved reporting of swine vaccination trials in the post-REFLECT statement publication period. J Swine Health Prod. (2019) 27:265–77. doi: 10.54846/jshap/1125

Crossref Full Text | Google Scholar

32. Brace, S, Taylor, D, and O’Connor, AM. The quality of reporting and publication status of vaccines trials presented at veterinary conferences from 1988 to 2003. Vaccine. (2010) 28:5306–14. doi: 10.1016/j.vaccine.2010.05.041

PubMed Abstract | Crossref Full Text | Google Scholar

33. Totton, SC, Cullen, JN, Sargeant, JM, and O’Connor, AM. The reporting characteristics of bovine respiratory disease clinical intervention trials published prior to and following publication of the REFLECT statement. Prev Vet Med. (2017) 150:117–25. doi: 10.1016/j.prevetmed.2017.12.015

Crossref Full Text | Google Scholar

34. Sargeant, JM, and O’Connor, AM. Issues of reporting in observational studies in veterinary medicine. Prev Vet Med. (2014) 113:323–30. doi: 10.1016/j.prevetmed.2013.09.004

PubMed Abstract | Crossref Full Text | Google Scholar

35. Horby, PW, Laurie, KL, Cowling, BJ, Engelhardt, OG, Jose, KS, Ramirez, JLS, et al. CONSISE statement on the reporting of Seroepidemiologic studies for influenza (ROSES-I statement): an extension of the STROBE statement. Influenza Other Respir Viruses. (2017) 11:2–14. doi: 10.1111/irv.12411

Crossref Full Text | Google Scholar

36. O’Connor, A, Totton, S, Winder, C, Holtkamp, D, and Sargeant, J. TRaiTS: template for reporting of trials in short format – swine examples. J Swine Health Prod. (2021) 29:327–33. doi: 10.54846/jshap/1247

Crossref Full Text | Google Scholar

37. Lim, WW, Leung, NHL, Sullivan, SG, Tchetgen Tchetgen, EJ, and Cowling, BJ. Distinguishing causation from correlation in the use of correlates of protection to evaluate and develop influenza vaccines. Am J Epidemiol. (2020) 189:185–92. doi: 10.1093/aje/kwz227

PubMed Abstract | Crossref Full Text | Google Scholar

38. Hudgens, MG, Gilbert, PB, and Self, SG. Endpoints in vaccine trials. Stat Methods Med Res. (2004) 13:89–114. doi: 10.1191/0962280204sm356ra

PubMed Abstract | Crossref Full Text | Google Scholar

39. Plotkin, SA. Vaccines: correlates of vaccine-induced immunity. Clin Infect Dis. (2008) 47:401–9. doi: 10.1086/589862

PubMed Abstract | Crossref Full Text | Google Scholar

40. Qin, L, Gilbert, PBB, Corey, L, McElrath, MJJ, and Self, SGG. A framework for assessing immunological correlates of protection in vaccine trials. J Infect Dis. (2007) 196:1304–12. doi: 10.1086/522428

PubMed Abstract | Crossref Full Text | Google Scholar

41. Weir, JP, and Gruber, MF. An overview of the regulation of influenza vaccines in the United States. Influenza Other Respir Viruses. (2016) 10:354–60. doi: 10.1111/irv.12383

PubMed Abstract | Crossref Full Text | Google Scholar

42. WHO. Guidelines on clinical evaluation of vaccines: regulatory expectations. WHO technical report. Geneva; (2004). Report No.: WHO/BS/2016.2287. Available at: http://www.who.int/biologicals/publications/trs/areas/vaccines/clinical_evaluation/en/ (Accessed March 14, 2020).

Google Scholar

43. CBER, FDA C. Guidance for industry-clinical data needed to support the licensure of seasonal inactivated influenza vaccines (2007). 15 Available at: http://www.fda.gov/cber/guidelines.htm. (Accessed November 22, 2020).

Google Scholar

44. Knight-Jones, TJD, Edmond, K, Gubbins, S, and Paton, DJ. Veterinary and human vaccine evaluation methods. Proc R Soc B Biol Sci. (1784) 281:20132839. doi: 10.1098/rspb.2013.2839

PubMed Abstract | Crossref Full Text | Google Scholar

45. Innis, BL, Berlanda Scorza, F, Blum, JS, Jain, VK, Older Aguilar, A, Post, DJ, et al. Meeting report: convening on the influenza human viral challenge model for universal influenza vaccines, part 1: value; challenge virus selection; regulatory, industry and ethical considerations; increasing standardization, access and capacity. Vaccine. (2019) 37:4823–9. doi: 10.1016/j.vaccine.2019.06.080

PubMed Abstract | Crossref Full Text | Google Scholar

46. Government of Canada. Guideline for autogenous veterinary biologics. (2022). Health of Animals Act and Regulations-Veterinary biologics guideline 3.13E. Available at: https://www.inspection.gc.ca/animal-health/veterinary-biologics/guidelines-forms/3-13e/eng/1328594554673/1328594626360 (accessed Mar 2, 2022).

Google Scholar

47. Amanna, IJ, Messaoudi, I, and Slifka, MK. Protective immunity following vaccination: how is it defined? Hum Vaccin. (2008) 4:316–9. doi: 10.4161/hv.4.4.5751

PubMed Abstract | Crossref Full Text | Google Scholar

48. Plotkin, SA. Correlates of protection induced by vaccination. Clin Vaccine Immunol. (2010) 17:1055–65. doi: 10.1128/CVI.00131-10

PubMed Abstract | Crossref Full Text | Google Scholar

49. Sheets, R, and Knezevic, I. Expert committee on biological standardization-human challenge trials for vaccine development: regulatory considerations. Geneva, Switzerland: World Health Organization. (2016) Available at: https://cdn.who.int/media/docs/default-source/immunization/sage/2017/sage-meeting-of-april-2017/background-docs/session-report-from-other-advisory-committees-on-immunization/6.-human-challenge-trials-for-vaccine-development--regulatory-considerations-pdf-388kb.pdf?sfvrsn=e05ea203_6 (Accessed November 22, 2020).

Google Scholar

50. Sandbulte, M, Spickler, A, Zaabel, P, and Roth, J. Optimal use of vaccines for control of influenza a virus in swine. Vaccines (Basel). (2015) 3:22–73. doi: 10.3390/vaccines3010022

PubMed Abstract | Crossref Full Text | Google Scholar

51. Moore, KA, Ostrowsky, JT, Kraigsley, AM, Mehr, AJ, Bresee, JS, Friede, MH, et al. A Research and Development (R & D) roadmap for influenza vaccines: looking toward the future. Vaccine. (2021) 39:6573–84. doi: 10.1016/j.vaccine.2021.08.010

PubMed Abstract | Crossref Full Text | Google Scholar

52. Holzer, B, Martini, V, Edmans, M, and Tchilian, E. T and B cell immune responses to influenza viruses in pigs. Front Immunol. (2019) 10:1–10. doi: 10.3389/fimmu.2019.00098

PubMed Abstract | Crossref Full Text | Google Scholar

53. Gianchecchi, E, Torelli, A, and Montomoli, E. The use of cell-mediated immunity for the evaluation of influenza vaccines: an upcoming necessity. Hum Vaccin Immunother. (2019) 15:1021–30. doi: 10.1080/21645515.2019.1565269

PubMed Abstract | Crossref Full Text | Google Scholar

54. Petrie, JG, Ohmit, SE, Johnson, E, Cross, RT, and Monto, AS. Efficacy studies of influenza vaccines: effect of end points used and characteristics of vaccine failures. J Infect Dis. (2011) 203:1309–15. doi: 10.1093/infdis/jir015

PubMed Abstract | Crossref Full Text | Google Scholar

55. Yewdell, JW, and Santos, JJS. Original antigenic sin: how original? How sinful? Cold Spring Harb Perspect Med. (2021) 11:1–16. doi: 10.1101/cshperspect.a038786

Crossref Full Text | Google Scholar

56. Hartshorn, KL. Innate immunity and influenza a virus pathogenesis: lessons for COVID-19. Front Cell Infect Microbiol. (2020) 10:1–17. doi: 10.3389/fcimb.2020.563850

PubMed Abstract | Crossref Full Text | Google Scholar

57. Smatti, MK, Al Thani, AA, and Yassine, HM. Viral-induced enhanced disease illness. Front Microbiol. (2018) 9:9. doi: 10.3389/fmicb.2018.02991

PubMed Abstract | Crossref Full Text | Google Scholar

58. Ramakrishnan, B, Viswanathan, K, Tharakaraman, K, Dančík, V, Raman, R, Babcock, GJ, et al. A structural and mathematical modeling analysis of the likelihood of antibody-dependent enhancement in influenza. Trends Microbiol. (2016) 24:933–43. doi: 10.1016/j.tim.2016.09.003

PubMed Abstract | Crossref Full Text | Google Scholar

59. Halstead, SB. Vaccine-associated enhanced viral disease: implications for viral vaccine development. BioDrugs. (2021) 35:505–15. doi: 10.1007/s40259-021-00495-6

PubMed Abstract | Crossref Full Text | Google Scholar

60. Khurana, S, Loving, CL, Manischewitz, J, King, LR, Gauger, PC, Henningson, J, et al. Vaccine-induced anti-HA2 antibodies promote vvirus fusion and enhance influenza virus respiratory disease. Sci Transl Med. (2013) 5:200ra114. doi: 10.1126/scitranslmed.3006366

PubMed Abstract | Crossref Full Text | Google Scholar

61. Gauger, PC, Loving, CL, Lager, KM, Janke, BH, Kehrli, ME, Roth, JA, et al. Vaccine-associated enhanced respiratory disease does not interfere with the adaptive immune response following challenge with pandemic a/h1n1 2009. Viral Immunol. (2013) 26:314–21. doi: 10.1089/vim.2013.0018

PubMed Abstract | Crossref Full Text | Google Scholar

62. McIlwain, DR, Chen, H, Rahil, Z, Bidoki, NH, Jiang, S, Bjornson, Z, et al. Human influenza virus challenge identifies cellular correlates of protection for oral vaccination. Cell Host Microbe. (2021) 29:1828–1837.e5. doi: 10.1016/j.chom.2021.10.009

PubMed Abstract | Crossref Full Text | Google Scholar

63. Bertho, N, and Meurens, F. The pig as a medical model for acquired respiratory diseases and dysfunctions: an immunological perspective. Mol Immunol. (2020) 135:254–67. doi: 10.1016/j.molimm.2021.03.014

PubMed Abstract | Crossref Full Text | Google Scholar

64. Lunney, JK, Van Goor, A, Walker, KE, Hailstock, T, Franklin, J, and Dai, C. Importance of the pig as a human biomedical model. Sci Transl Med. (2021) 13:1–20. doi: 10.1126/scitranslmed.abd5758

PubMed Abstract | Crossref Full Text | Google Scholar