Evaluating Public Health Interventions: A Neglected Area in Health Technology Assessment

Introduction: Public health (PH) interventions are crucial for ensuring sustainable healthcare services. Nevertheless, they represent a neglected area in the field of health technology assessment (HTA) due to various methodological issues and their complex design that goes beyond clinical setting. The present study provides an environmental scan of HTA initiatives related to the assessment of PH technologies on a global level.

Methods: We conducted a cross-sectional survey among 85 HTA-related European and international societies, health bodies, and networks from September 2018 to January 2019. The questionnaire contained four sections and 18 questions regarding activities related to the evaluation of PH technologies, information on existing PH technologies, and methodologies of assessment as well as barriers and facilitators to reaching a decision and implementing a PH technology.

Results: Among 52 survey responses, the majority of the respondents came from European countries (35%), followed by North American (27%), and South American (19%) countries. The main type of organizations covered by our survey included HTA agencies, public administrations, and research institutes. Seventy-one % of the institutions reported engagement in any aspect of HTA in the area of PH (N = 37). Among those, 81% evaluated less than 5 PH technologies from 2013 to 2018. The most common barriers for reaching a decision on PH technologies were lack of data, conflicting stakeholder priorities, and methodological issues. A total of 76 PH interventions were reported, and most cited initiatives were related to chronic disease screening, prevention of infectious diseases, and maternal, prenatal, and neonatal screening.

Conclusion: Our survey reported a rather limited involvement of HTA in the evaluation of PH technologies. In particular, an evaluation of behavioral and lifestyle interventions remains extremely rare. The implementation of collaborative HTA approaches in the setting of PH practice and policy needs to be prioritized and further strengthened. Moreover, ensuring reliable data structures and consolidation of HTA methods for the evaluation of PH technologies will be crucial for tackling the enormous burden of non-communicable diseases in societies.

Introduction

Non-communicable diseases are the biggest drivers of global mortality, contributing to over 70% of total deaths in 2017. The vast majority of premature deaths occur in low- and middle-income countries in part due to behavioral risk factors such as increasing rates of tobacco and alcohol consumption, unbalanced diets, and physical inactivity (1).

Parallel to these devastating figures, the world population health is facing numerous additional threats including outbreaks of vaccine-preventable diseases, increasing antimicrobial resistance, population aging, as well as air pollution and climate change (2–4).

The sustainability of healthcare systems may be supported by investing more in health promotion, disease prevention, and early diagnosis rather than in disease treatment (5). With the most recent technological advancements as well as citizen empowerment, there is a tendency to move away from the “one size fits all” approach and to develop personalized approaches that would place citizens at the core of health systems (6).

Public health (PH) practitioners and institutions have a crucial role in tackling these challenges by bringing together health professionals, service providers, policymaking authorities, and governmental agencies to strive for healthy communities while ensuring social justice and equity.

Health technology assessment (HTA), as a multidisciplinary process, applies systematic procedures when evaluating health technologies in order to support and guide health policy decision making. HTA comprehensively appraises health technologies, focusing both on their direct effects, and indirect consequences (7).

Traditionally, the vast majority of HTAs have been concentrating on the clinical area, as well as on pharmaceuticals and/or medical devices as the main target. Lavis et al. provided an inventory of HTAs across nine diverse HTA organizations, conducted from 2003 to 2006, highlighting that only 5% of health technologies were PH interventions (8).

A systematic evaluation of effectiveness can be easily applied in the field of clinical interventions, while the complexity of PH interventions in the real-world setting still poses methodological challenges (9–11). A recent systematic review about HTAs of PH interventions reported a limited number of guidelines in this field. The authors also observed numerous, and often unjustified, changes to recommendations such as adaptations in the processes of quality evaluation, assessment of applicability, and integration of qualitative and quantitative evidence (12, 13). These methodological tweaks were mainly linked to the fact that complex PH interventions implement diverse methodologies that often fall outside a controlled experimental setup such as those processed in randomized controlled trials. However, this could not be a justification for the scarce number of PH technologies assessed by HTA bodies.

We performed a cross-sectional survey among international organizations, networks, and societies involved in HTA in order to: (1) analyze the magnitude of their involvement in the evaluation of PH technologies, (2) provide specific information on existing PH technologies as well as methodologies of assessment, and (3) understand barriers to assessing/reaching a decision on and implementing a PH intervention. For the purpose of this survey, PH was defined as all organized measures (whether public or private) to prevent disease, promote health, and prolong life among the population as a whole, involving a collaborative effort by all parts of the health sector working to ensure the well-being of society through comprehensive prevention, treatment, care, and support (14).

Materials and Methods

Survey Development

Three authors (JS, ILW, and IGI), executive members of the HTAi Interest Group Initiative for Public Health Outcomes Research and Measurement (INPHORM), developed a draft questionnaire in July 2018 following a series of discussion-based meetings and subsequent exchanges until consensus was reached. The draft survey was sent for additional consultation to the HTAi members present at the INPHORM business meeting during the HTAi Annual Meeting in Vancouver, Canada.

The final survey consisted of 18 questions across four different sections, including contact information, activities related to evaluation of PH technologies, PH intervention candidates, and questions about the process of reaching a decision on a PH technology (Supplementary Table 1). Briefly, the survey collected data on what kind of PH technologies and/or interventions had been assessed in the last 5 years as well as specific details of the technology/intervention. Moreover, the questionnaire encompassed a mapping of methodologies used in the assessment and of enablers and barriers to assessing, adopting, or delisting the intervention.

Survey Distribution and Data Collection

Survey Software

The survey was implemented using an online survey management system (15) that ensured the confidentiality of the answers. An email explaining the purpose of the questionnaire and indicating the deadline for submitting responses was sent with the link to each of the participants,. The survey was distributed among the member lists of diverse HTAi partners: international societies, health bodies, and networks, including Health Technology Assessment international (HTAi), The International Information Network on new or emerging, appropriate use and re-assessment needed Health Technologies (EuroScan International Network e.V.), the European network for Health Technology Assessment (EUnetHTA), the International Network of Agencies for Health Technology Assessment (INAHTA), the National Health Service of the United Kingdom (NHS-UK), the York Health Economics Consortium (YHEC), the Institute for Quality and Efficiency in HealthCare (IQWiG), the Haute Autorité de Santé (HAS), the National Institute for Public Health and the Environment (RIVM), the European Public Health Association (EUPHA-HTA section), the Red Española de Agencias de Evaluación de Tecnologías Sanitarias (REDETS), the Red de Evaluación de Tecnologia en Salud de las Américas (RedETSA), and the Health Technology Assessment Asia Network (HTAsialink). It was sent out to a total of 85 recipients and was open from September 2018 to January 2019. Data extraction of the responses to the questionnaire was conducted by one author (ERR) and was cross-checked for consistency by two additional authors (JS and ILW). Any emerging discrepancies were resolved through discussion.

Statistical Analysis

Descriptive statistics were calculated to summarize the sample. The statistical analysis was conducted in IBM SPSS Statistics 25 software and STATA version 12 software.

ICHI Encoding

In addition to the statistical analysis, we tested the feasibility of using the International Classification of Health Interventions (ICHI) for categorization of the survey outcomes. The ICHI is an open-access classification available via an online database hosted by DIMDI (16). The ICHI is part of the WHO-FIC reference classifications, with the aim to develop a common tool for reporting and analyzing health interventions for statistical, quality, and reimbursement purposes. Given the paramount importance of PH within all health structures worldwide, PH represents an integrated component of the ICHI.

The coding for the present study was performed through searching for the intervention titles by a single pair of authors (MW and SG). If the feasibility testing proved to be positive, we would conduct a follow-up project by involving additional reviewers and ensuring the independent coding process (work in progress).

Results

Description of the Respondents

In total, 52 respondents from all continents answered the survey. The majority came from European countries (34%), followed by North American (26.9%) and South American countries (19.2%). Equal numbers of HTA agencies, public administration institutions, and scientific research institutes responded to the survey (N = 11 each). The remaining respondents included nine academic, six industry, and four hospital institutions. These results are reported in Supplementary Figures 1, 2.

Public Health Intervention Candidates

Our survey yielded a total of 76 PH technologies reported by the respondent institutions. A wide variety of interventions was identified, targeting different populations and diseases. In general, the reported technologies could be roughly subdivided in the following groups: primary prevention (42.1%), secondary prevention (48.7%), tertiary prevention (5.3%), and others (policy, mixed, etc.) (3.9%). When sorting the answers according to health concerns, screening of chronic diseases showed the highest proportion of interventions (25%), followed by infectious diseases prevention, in particular vaccines and disease screening (21.1%), and maternal, prenatal, and neonatal screening initiatives (9.2%). The reported assessment of behavioral interventions was low both in the adult population (3.95%) as well as in children and adolescents (3.95%). Similar outcomes were found for environmental interventions such as tobacco cessation (3.9%) and mental health screening (3.9%).

The survey question on the most important reasons for conducting a PH assessment allowed several answers per respondent. The reason most frequently reported in this survey was the need to identify whether a PH intervention represents a better alternative to standard procedures (75%). Other important drivers were the need to evaluate the impact on change in healthcare practices (51.3%) and the impact of an intervention on healthcare budget/resources (48.7%) (data not shown).

Assessment of PH Technologies

Out of the 52 respondents to the survey, 37 institutions reported engaging in any aspect of HTA in the PH area (71%). Thirty-four of these organizations were involved in assessment activities, including reviewing and/or generating evidence of effectiveness or cost-effectiveness (Table 1). The majority of respondents stated that their institution is involved in assessing the medical aspects of the technology (N = 32), followed by economic considerations (N = 25), organizational impact (N = 23), and societal consequences (N = 23) (Table 2).

TABLE 1

Table 1. An overview of activities of 37 organizations engaged in PH technology assessment on (cost-)effectiveness (multiple-choice question).

TABLE 2

Table 2. Answers regarding aspects of the PH technology assessment reported by 34 organizations involved in the assessment activities (multiple-choice question).

Concerning the volume of evaluated PH technologies, 81% of the 37 institutions reported evaluating <5 PH technologies, while 27% of the institutions had evaluated only one PH technology since 2013. Various methodologies for assessing PH technologies were reported, the most common of which include HTA, budget-impact analyses, and multi-criteria decision analyses (Table 3).

TABLE 3

Table 3. Methods, frameworks, or tools reported by 37 organizations engaged in PH technology assessment (multiple-choice question).

To the question “is your organization currently involved in or planning to launch an evaluation of a PH technology?,” 42% of the 52 survey respondents replied positively.

Barriers to Reaching a Decision About Implementing a PH Technology

The most commonly reported barriers in reaching a decision on the adoption or delisting of a PH technology were lack of relevant data to conduct an assessment (54%), conflicting priorities among diverse stakeholders (43%), common methodological issues and lack of clear methodological frameworks to properly assess PH interventions through an HTA approach (32%), difficulty to assess the impact and to reallocate resources across and between programs or sectors of complex PH interventions (30%), and political challenges (30%) (Supplementary Table 2).

In regard to the barriers of implementation, 41% of the institutions reported not being engaged in decision implementation processes. Among the 22 institutions, the most common barriers involved in implementing decisions were clinicians' reluctance to change habits in their daily practice (24%), lack of staff and resources necessary to implement the intervention (22%), lack of funding for implementation (22%), perception that management priority is costing money while return on investment is not warranted (22%), and lack of perceived benefit related to the complex context of the technology/bundle of technologies (19%) (Supplementary Table 3).

Coding According to the ICHI

In many cases it was necessary to describe the intervention with appropriate keywords since the exact title was not defined in the ICHI. A certain training period was required because of the structure of the ICHI codes; technical handling of the platform is very workable after a short orientation period.

A precisely descriptive ICHI code could rarely be assigned to the executed intervention. Often codes had to be used which would approximate the interventions. In addition, there were cases in which interventions corresponded to different codes regarding “target” and “action”; therefore, a combination of these codes would have been useful.

Furthermore, the PH interventions were not always described precisely, which meant several codes could be considered. This notwithstanding, after encoding for each of the reported PH interventions, it appeared that several answers could be clustered in a relatively small number of categories of the ICHI (Table 4).

TABLE 4

Table 4. Answers coded with WHO International Classification of Health Interventions (ICHI) after a single-pair categorization (most mentioned categories: N > 3).

Discussion

The present survey involved 52 international HTA bodies and aimed to capture their engagement in the PH field as well as specific PH technology characteristics and methodologies of assessment. Around 70% of the responding institutions reported being engaged in PH assessments, mainly HTA agencies, research institutes, and public administration bodies.

The outcomes of this survey indicate that the assessment of PH interventions represents only a limited proportion of evaluations carried out by institutions engaged in HTA. Indeed over 80% of the institutions reported performing <5 PH technology evaluations since 2013.

Considering the PH intervention candidates, our survey reports a total of 76 PH technologies that had been evaluated by the institutions. Although the majority of the reported assessments focuses on primary and secondary prevention programs, it appears that in many cases the targeted populations are already benefiting from some form of clinical monitoring or care provisions, indicating that the prevention intervention approaches are more frequently delivered within the healthcare setting, thus not reaching the far higher number of concerned (at-risk) citizens who do not access healthcare services. Nevertheless, there is an increasing number of implemented initiatives that are linking primary care with sources of community support, but the evidence about their effectiveness and methodological rigor is limited (17). The key issue with building the evidence base for initiatives such as social prescribing as well as PH interventions in general lies in the fact that applying the golden standard of evidence (i.e., double-blinded randomized controlled trials) might not be adapted. Pragmatic trials, as a more naturalistic approach, might provide a better picture of effectiveness, while evidence summaries may include a realist synthesis as opposed to standard systematic review approaches (12, 18–20). This being said, it should be kept in mind that good evidence does not necessarily lead to good policy (21).

In our survey, <5% of the interventions focused on behavior. The evaluation of behavioral and lifestyle interventions remains extremely rare in spite of their importance for developing effective population health strategies and establishing health systems. This notwithstanding, PH researchers and other non-HTA professionals frequently study a variety of approaches for prevention interventions that directly target the general population (22, 23). However, associated recommendations and more generalized implementation of campaigns and interventions do not belong to their mainstream responsibilities. Moreover, very few behavioral trials have been registered and many of them lack transparent reporting (24). It is thus important to emphasize that rigorous conduct as well as transparent reporting need to be implemented.

The inadequacy of HTA from an overall PH perspective is also reported in literature (12, 25–27). One of the underlying reasons probably lays in the multi-dimensional character of PH, a field which is increasingly affected by a full range of extra-clinical circumstances (28, 29), including demographic transitions (30) and modifiable risk factors of disease development associated with unhealthy lifestyle habits in the general population (31–35). Lifestyle research has to do with socio-economic aspects including behavior sciences and drivers of individual choices; effective PH management therefore requires an integrated perspective on the health–disease continuum (36–40). The resulting uncertainties (41), the fact that benefits at the population level might not necessarily be synonymous to benefits for single individuals, as well as a lack of insight in the variability of the many underlying causal pathways (42) often lead to a de-prioritization of preventive interventions on the population level, emphasizing that collaborative approaches among different professionals (from PH practitioners to behavioral scientists, psychologists, community and social workers, etc.) will be crucial for implementing efficient and effective interventions once the decision has been made.

Not surprisingly, a median return on investment of PH interventions is rather high. A recent systematic review estimated that each pound invested in PH will lead to 14 pounds being returned to the health and social care economy. Yet again, many of PH interventions, in particular those associated with behavioral risk factors, do not get funded. One of the plausible explanations is that political incentives are lacking since the benefits of PH interventions usually arrive after the mandate of current politicians and policy makers (43). Furthermore, it could be related to a lack of interest due to the inexistence of promoters as it is the case of other technologies that are designed and marketed by companies with an obvious business interest in their marketing.

This might also explain—at least partly—the barriers for reaching a decision about a PH technology or the implementation of a PH intervention as mentioned by the respondents of our survey. Similar elements are reported in the recently published INAHTA viewpoint (44).

The distribution of our survey via the HTAi partners may have induced an underrepresentation of PH specialists and represent a limitation. Indeed, as mentioned above, HTA approaches are getting increasingly used by PH scientists as a recognized instrument for obtaining insight on various kinds of PH measures and their impact on health logistics/systems. Recent reports show that their studies are frequently supported by HTA organizations (45–48); this type of collaborations was not necessarily captured in the survey.

ICHI Content Suitability

Basically, the suitability of the ICHI could be determined and we were able to code the PH interventions. We found multiple ICHI codes matching the survey data, which was also related to the inductive approach of our survey. To reach a certain level of specificity, the ICHI classification could be used to design future surveys that integrate codes and definitions to questionnaires. However, this will likely lead to an extension of the survey and fewer responses of individuals. There were clear differences concerning the accuracy of fit. Nevertheless, there has been coding in all four categories of interventions. Differences were found in the three axes of each ICHI measure (target, action, and means). In many cases, the means were coded as “other and unspecified means.”

Representing the historically larger number of health professionals engaged in the clinical treatment of diseases, the ICHI classification also shows a strong clinical focus. In general, it was found that the ICHI criteria primarily involved functions (such as body regions or involved healthcare institutions) throughout general PH interventions which are often targeting diseases and specific diagnoses. Regarding the beta status of ICHI, the specifics of PH interventions might be integrated into future ICHI versions to support coordinated research and decisions in HTA worldwide.

Conclusions

Over the last decades, considerable efforts have been deployed to act against the increasing burden of PH concerns overall. There is now enough hindsight to realize that many of the implemented strategies and interventions do not allow to put an end to the relentless spread of the so-called civilization diseases and recent outbreaks of vaccine-preventable diseases in a decisive manner.

HTA is a young discipline that originated from the need to support decision making on value in healthcare and, for the moment, the original disease-centered HTA focus still holds a dominating place, often at the expense of a more general health maintenance priority setting on the long term. This survey underlines a certain inadequacy of HTA from an overall PH perspective. Reshaping the current public health HTA methods and related practices of policy priority setting will be crucial for implementing efficient and effective interventions. The quickly expanding use of the HTA concepts by professionals and stakeholders from different horizons holds great promise for ameliorations in PH-oriented research and strategies across the different scientific and policy territories in spite of the challenges this poses (13, 49–51).

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

Ethical approval for this study was not required in accordance with national legislation and institutional requirements due to fact that the data presented are anonymous, present no potential for identification of specific individuals with the dissemination of the research results, and are not considered to be sensitive or confidential in nature.

Author Contributions

JS, IG-I, and IL-W contributed to the conception and design of the study. IG-I and ER organized the distribution of the survey. JS and ER extracted the survey data and performed the statistical analysis. JS, ER, and IL-W performed the initial classification of PH interventions. MW and SG performed the coding of PH interventions according to ICHI and wrote the sections of the manuscript. JS and IL-W wrote the first draft of the manuscript. All the authors contributed to manuscript revision as well as read and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank Dr. Grace Hampson and Dr. Alexia Campbell Burton for their kind feedback on the survey content. The authors gratefully acknowledge the HTAi society for taking charge of the journal's publication fees.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2020.00106/full#supplementary-material

References

1. Roth GA, Abate KH, Abate D, Abay SM, Abbafati C, Abbasi N, et al. Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet. (2018) 392:1736–88. doi: 10.1016/S0140-6736(18)32203-7

CrossRef Full Text | Google Scholar

2. World Health Organization. The Thirteenth General Programme of Work, 2019–2023. (2018). Available online at: https://apps.who.int/iris/bitstream/handle/10665/324775/WHO-PRP-18.1-eng.pdf (accessed December 20, 2019).

Google Scholar

3. Peeri NC, Shrestha N, Rahman MS, Zaki R, Tan Z, Bibi S, et al. The SARS, MERS and novel coronavirus (COVID-19) epidemics, the newest and biggest global health threats: what lessons have we learned? Int J Epidemiol. (2020) dyaa033. doi: 10.1093/ije/dyaa033. [Epub ahead of print].

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Hasan S, Ahmad SA, Masood R, Saeed S. Ebola virus: A global public health menace: A narrative review. J Family Med Prim Care. (2019) 8:2189–2201. doi: 10.4103/jfmpc.jfmpc_297_19

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Crisp N. What would a sustainable health and care system look like? BMJ. (2017) 358:j3895. doi: 10.1136/bmj.j3895

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Ricciardi W, Boccia S. New challenges of public health: bringing the future of personalised healthcare into focus. Eur J Public Health. (2017) 27:36–9. doi: 10.1093/eurpub/ckx164

PubMed Abstract | CrossRef Full Text | Google Scholar

7. HtaGlossary.net | HomePage. Available at: http://htaglossary.net/HomePage (accessed december 20, 2019).

Google Scholar

8. Lavis JN, Wilson MG, Grimshaw JM, Haynes RB, Ouimet M, Raina P, et al. Supporting the use of health technology assessments in policy making about health systems. Int J Technol Assess Health Care. (2010) 26:405–14. doi: 10.1017/S026646231000108X

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Burchett HED, Blanchard L, Kneale D, Thomas J. Assessing the applicability of public health intervention evaluations from one setting to another: A methodological study of the usability and usefulness of assessment tools and frameworks. Heal Res Policy Syst. (2018) 16:88. doi: 10.1186/s12961-018-0364-3

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Petticrew M, Chalabi Z, Jones DR. To RCT or not to RCT: deciding when ‘more evidence is needed’ for public health policy and practice. J Epidemiol Community Health. (2012) 66:391–6. doi: 10.1136/jech.2010.116483

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. (2002) 56:119–27. doi: 10.1136/jech.56.2.119

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mathes T, Antoine S-L, Prengel P, Bühn S, Polus S, Pieper D. Health technology assessment of public health interventions: a synthesis of methodological guidance. Int J Technol Assess Health Care. (2017) 33:135–46. doi: 10.1017/S0266462317000228

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Mathes T, Willms G, Polus S, Stegbauer C, Messer M, Klingler C, et al. Health technology assessment of public health interventions: an analysis of characteristics and comparison of methods-study protocol. Syst Rev. (2018) 7:79. doi: 10.1186/s13643-018-0743-4

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Marks L, Hunter DJ. Strengthening Public Health Capacity and Services in Europe. WHO Publ. (2011). Available online at: http://www.euro.who.int/pubrequest (accessed December 20, 2019).

Google Scholar

15. Encuestas online. Software Encuesta - Crea y Envia Cuestionarios Facilmente. Available online at: https://www.encuestafacil.com/ (accessed December 20, 2019).

Google Scholar

16. International Classification of Health Interventions. (ICHI). Beta-2 2019. Available online at: https://mitel.dimi.uniud.it/ichi/ (accessed December 20, 2019).

Google Scholar

17. Bickerdike L, Booth A, Wilson PM, Farley K, Wright K. Social prescribing: less rhetoric and more reality. A systematic review of the evidence. BMJ Open. (2017) 7:e013384. doi: 10.1136/bmjopen-2016-013384

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review - a new method of systematic review designed for complex policy interventions. J Heal Serv Res Policy. (2005) 10:21–34. doi: 10.1258/1355819054308530

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Jani A, Pitini E, Jungmann S, Adamo G, Conibear J, Mistry P. A social prescriptions formulary: bringing social prescribing on par with pharmaceutical prescribing. J R Soc Med. (2019) 112:498–502. doi: 10.1177/0141076819877555

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Christian JB, Brouwer ES, Girman CJ, Bennett D, Davis KJ, Dreyer NA. Masking in pragmatic trials: who, what, and when to blind. Ther Innov Regul Sci. (2019) 2:216847901984312–2168479019843129. doi: 10.1177/2168479019843129

CrossRef Full Text | Google Scholar

21. Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ. (2004) 82:724–32.

PubMed Abstract | Google Scholar

22. Lazarus JV, Ekstedt M, Marchesini G, Mullen J, Novak K, Pericàs JM, et al. A cross-sectional study of the public health response to non-alcoholic fatty liver disease in Europe. J Hepatol. (2020) 72:14–24. doi: 10.1016/j.jhep.2019.08.027

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Mullee A, Romaguera D, Pearson-Stuttard J, Viallon V, Stepien M, Freisling H, et al. Association between soft drink consumption and mortality in 10 European Countries. JAMA Intern Med. (2019) 179:1479–90. doi: 10.1001/jamainternmed.2019.2478

CrossRef Full Text | Google Scholar

24. ClinicalTrials.gov. Trends, Charts, and Maps - ClinicalTrials.gov. Web. (2018). Available at: https://clinicaltrials.gov/ct2/resources/trends#TypesOfRegisteredStudies (accessed December 20, 2019).

Google Scholar

25. Cacciatore P, Specchia ML, Solinas MG, Ricciardi W, Damiani G. The organizational domain in HTA reports: towards a technology-oriented assessment. Eur J Public Health. (2019) 2019:ckz173. doi: 10.1093/eurpub/ckz173

CrossRef Full Text | Google Scholar

26. Edwards RT, Charles JM, Lloyd-Williams H. Public health economics: a systematic review of guidance for the economic evaluation of public health interventions and discussion of key methodological issues. BMC Public Health. (2013) 13:1001. doi: 10.1186/1471-2458-13-1001

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Vukovic V, Favaretti C, Ricciardi W, de Waure C. Health technology assessment evidence on e-health/m-health technologies: evaluating the transparency and thoroughness. Int J Technol Assess Health Care. (2018) 34:87–96. doi: 10.1017/S0266462317004512

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Di Cesare M, Sorić M, Bovet P, Miranda JJ, Bhutta Z, Stevens GA, et al. The epidemiological burden of obesity in childhood: a worldwide epidemic requiring urgent action. BMC Med. (2019) 17:212. doi: 10.1186/s12916-019-1449-8

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Graham H, White PCL. Social determinants and lifestyles: integrating environmental and public health perspectives. Public Health. (2016) 141:270–8. doi: 10.1016/j.puhe.2016.09.019

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Juzwishin D, McNeil H, Ahn J, Chen Y, Cicchetti A, Kume N, et al. Aging and health technology assessment: an idea whose time has come. Int J Technol Assess Health Care. (2018) 34:442–6. doi: 10.1017/S0266462318000600

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Leng G, Adan RAH, Belot M, Brunstrom JM, De Graaf K, Dickson SL, et al. “The determinants of food choice,” in Proceedings of the Nutrition Society (Cambridge: Cambridge University Press), (2017) 76:316–27. doi: 10.1017/S002966511600286X

CrossRef Full Text | Google Scholar

32. Dekker LH, de Borst MH, Meems LMG, de Boer RA, Bakker SJL, Navis GJ. The association of multimorbidity within cardio-metabolic disease domains with dietary patterns: a cross-sectional study in 129 369 men and women from the Lifelines cohort. PLoS ONE. (2019) 14:e0220368. doi: 10.1371/journal.pone.0220368

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Sanner T, Grimsrud TK. Nicotine: carcinogenicity and effects on response to cancer treatment - a review. Front Oncol. (2015) 5:196. doi: 10.3389/fonc.2015.00196

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Wang DD, Li Y, Afshin A, Springmann M, Mozaffarian D, Stampfer MJ, et al. Global improvement in dietary quality could lead to substantial reduction in premature death. J Nutr. (2019) 149:1065–74. doi: 10.1093/jn/nxz010

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Aceijas C, Waldhäusl S, Lambert N, Cassar S, Bello-Corassa R. Determinants of health-related lifestyles among university students. Perspect Public Health. (2017) 137:227–36. doi: 10.1177/1757913916666875

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Bousquet J, Pham-Thi N, Bedbrook A, Agache I, Annesi-Maesano I, Ansotegui I, et al. Next-generation care pathways for allergic rhinitis and asthma multimorbidity: a model for multimorbid non-communicable diseases—Meeting Report. (Part 2). J Thoracic Dis. (2019) 2019:4072–84. doi: 10.21037/jtd.2019.09.38

CrossRef Full Text | Google Scholar

37. Mozaffarian D, Mande J, Micha R. Food is medicine-the promise and challenges of integrating food and nutrition into health care. JAMA Intern Med. (2019) 179:793–5. doi: 10.1001/jamainternmed.2019.0184

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Sagner M, McNeil A, Puska P, Auffray C, Price ND, Hood L, et al. The P4 health spectrum – a predictive, preventive, personalized and participatory continuum for promoting healthspan. Prog Cardiovasc Dis. (2017) 59:506–21. doi: 10.1016/j.pcad.2016.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

39. van Ommen B, Wopereis S, van Empelen P, van Keulen HM, Otten W, Kasteleyn M, et al. From diabetes care to diabetes cure-the integration of systems biology, ehealth, and behavioral change. Front Endocrinol. (2018) 8:381. doi: 10.3389/fendo.2017.00381

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Schwartz S, Diez-Roux R. Commentary: causes of incidence and causes of cases - a durkheimian perspective on Rose. Int J Epidemiol. (2001) 30:435–9. doi: 10.1093/ije/30.3.435

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Stewart A, Hursthouse A. Environment and human health: the challenge of uncertainty in risk assessment. Geosciences. (2018) 8:24. doi: 10.3390/geosciences8010024

CrossRef Full Text | Google Scholar

42. Braveman P, Gottlieb L. The social determinants of health: it's time to consider the causes of the causes. (2014) 129(Suppl. 2):19–31. doi: 10.1177/00333549141291S206

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Masters R, Anwar E, Collins B, Cookson R, Capewell S. Return on investment of public health interventions: a systematic review. J Epidemiol Community Health. (2017) 71:827–34. doi: 10.1136/jech-2016-208141

PubMed Abstract | CrossRef Full Text | Google Scholar

44. O'Rourke B, Werkö SS, Merlin T, Huang LY, Schuller T. The ‘Top 10’ challenges for health technology assessment: INAHTA viewpoint. Int J Technol Assess Health Care. (2019) 36:1–4. doi: 10.1017/S0266462319000825

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Cobiac LJ, Scarborough P, Kaur A, Rayner M. The eatwell guide: modelling the health implications of incorporating new sugar and fibre guidelines. PLoS ONE. (2016) 11:e0167859. doi: 10.1371/journal.pone.0167859

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Mytton OT, Tainio M, Ogilvie D, Panter J, Cobiac L, Woodcock J. The modelled impact of increases in physical activity: the effect of both increased survival and reduced incidence of disease. Eur J Epidemiol. (2017) 32:235–50. doi: 10.1007/s10654-017-0235-1

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Briggs ADM, Wolstenholme J, Scarborough P. Estimating the cost-effectiveness of salt reformulation and increasing access to leisure centres in England, with PRIMEtime CE model validation using the AdViSHE tool. BMC Health Serv Res. (2019) 19:489. doi: 10.1186/s12913-019-4292-x

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Martineau AR, Jolliffe DA, Greenberg L, Aloia JF, Bergman P, Dubnov-Raz G, et al. Vitamin D supplementation to prevent acute respiratory infections: individual participant data meta-analysis. Health Technol Assess. (2019) 23:1–44. doi: 10.3310/hta23020

PubMed Abstract | CrossRef Full Text | Google Scholar

49. White C, Sanders Schmidler G, Butler M, Wang Z, Robinson K, Berkman N, et al. Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking. Research White Paper. Rockville, MD: Agency for Healthcare Research and Quality; Publication No. 17(18)-EHC035-EF (2017).

Google Scholar

50. Phillips SA, Martino S, Arena R. Research opportunities and challenges in the era of healthy living medicine: unlocking the potential. Prog Cardiovasc Dis. (2017) 59:498–505. doi: 10.1016/j.pcad.2017.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Witter S, Palmer N, Balabanova D, Mounier-Jack S, Martineau T, Klicpera A, et al. Health system strengthening—Reflections on its meaning, assessment, and our state of knowledge. Int J Health Plann Manage. (2019) 34:e1980–9. doi: 10.1002/hpm.2882

PubMed Abstract | CrossRef Full Text | Google Scholar