Claire Kerins1*
Colette Kelly1†
Caitlin M. Reardon2†
Catherine Houghton3
Elaine Toomey4,5
Catherine B. Hayes6Fiona Geaney7Ivan J. Perry7Jenny McSharry8
Sheena McHugh7- 1Discipline of Health Promotion, School of Health Sciences, National University of Ireland Galway, Galway, Ireland
- 2VA Center for Clinical Management Research, VA Ann Arbor Healthcare System, Ann Arbor, MI, United States
- 3School of Nursing and Midwifery, National University of Ireland Galway, Galway, Ireland
- 4Faculty of Education and Health Sciences, School of Allied Health, University of Limerick, Limerick, Ireland
- 5Health Research Institute, University of Limerick, Limerick, Ireland
- 6Public Health and Primary Care, Institute of Population Health, School of Medicine, Trinity College Dublin, Dublin, Ireland
- 7School of Public Health, University College Cork, Cork, Ireland
- 8Health Behaviour Change Research Group, School of Psychology, National University of Ireland Galway, Galway, Ireland
Background: Labelling menus with nutrition information has increasingly become an important obesity policy option. While much research to-date has focused on determining its effectiveness, few studies report the extent to which menu labelling is implemented as designed. The aim of this study was to explore factors influencing fidelity to a calorie posting policy in Irish acute public hospitals.
Methods: A mixed methods sequential explanatory study design was employed, with a nested case study for the qualitative component. Quantitative data on implementation fidelity at hospitals were analysed first and informed case sampling in the follow-on qualitative phase. Maximum variation sampling was used to select four hospitals with high and low levels of implementation and variation in terms of geographic location, hospital size, complexity of care provided and hospital type. Data were collected using structured observations, unstructured non-participant observations and in-depth semi-structured interviews. The Consolidated Framework for Implementation Research guided qualitative data collection and analysis. Using framework analysis, factors influencing implementation were identified. A triangulation protocol was used to integrate fidelity findings from multiple sources. Data on influencing factors and fidelity were then combined using joint displays for within and cross-case analysis.
Results: Quantitative fidelity data showed seven hospitals were categorised as low implementers and 28 hospitals were high implementers of the policy. Across the four hospitals selected as cases, qualitative analysis revealed factors influencing implementation and fidelity were multiple, and operated independently and in combination. Factors were related to the internal hospital environment (e.g., leadership support, access to knowledge and information, perceived importance of calorie posting implementation), external hospital environment (e.g., national policy, monitoring), features of the calorie posting policy (e.g., availability of supporting materials), and the implementation process (e.g., engaging relevant stakeholders). Integrated analysis of fidelity indicated a pattern of partial adherence to the calorie posting policy across the four hospitals. Across all hospitals, there was a consistent pattern of low adherence to calorie posting across all menu items on sale, low adherence to calorie information displayed per standard portion or per meal, low adherence to standardised recipes/portions, and inaccurate calorie information.
Conclusion: Efforts to maximise fidelity require multi-level, multi-component strategies in order to reduce or mitigate barriers and to leverage facilitators. Future research should examine the relative importance of calorie posting determinants and the association between implementation strategies and shifts in fidelity to intervention core components.
Background
The World Health Organisation estimates that, worldwide, the prevalence of obesity has reached epidemic proportions (1). While there are multifactorial drivers (2), research shows eating outside the home may play a role in the current obesity epidemic (3–6). Eating outside the home has become more frequent (7–9), and is associated with higher energy content of meals (10–13) and consumer underestimation of calorie content (14, 15). In an effort to address this, labelling menus with nutrition information (such as the calorie content of menu items) at the point of sale has become a popular policy option (16–18). A number of countries and regions around the world have menu labelling on a voluntary or mandatory basis for food service businesses (19–23). The process for menu labelling policy development across food businesses is not routinely documented. Furthermore, a number of workplace and healthcare organisations have introduced menu labelling policies at national and local levels (24–26). Evidence from two recent systematic reviews suggests labelling has positive effects on consumer dietary intake (27, 28) and industry practises (i.e., reformulation of menu items) (28).
While much research has been concerned with determining the effectiveness of menu labelling (27, 28), few studies report the extent to which menu labelling is implemented as designed (29–33). Of these, the predominant focus has been in the restaurant setting, with issues identified concerning poor uptake (29, 30), lack of adherence to standardised recipes and portions, and inaccurate calorie information being displayed (31–33). A recent systematic review identified factors influencing implementation in food service businesses (34). Factors were related to the external context of food businesses (e.g., consumers, legislation), internal setting of food businesses (e.g., compatibility, available information and resources) and features of the menu labelling intervention (e.g., perceived benefits, cost) (34). Few studies have explored the implementation process in the workplace (35, 36) and healthcare setting (25). One such study in the workplace found variation in the proportion of canteen menu items labelled with calories (50–99%) and issues relating to accuracy of this information (35). Thus far, no study has examined adherence to menu labelling policies in the healthcare setting.
With employees now spending longer periods of time in the work environment (37–39) and consuming meals prepared in the workplace canteen (40, 41), the health and well-being of employees has moved to the forefront of organisational agendas. In particular, the healthcare industry, being one of the largest employers in many countries, increasingly recognises their leadership role in serving as public health role models as well as health promotion advocates (42). In an effort to promote staff health and wellbeing, and to act as an exemplar across the public service, the Irish Health Service Executive (HSE) introduced a calorie posting policy in 2015 (43). The HSE manages the delivery of all public health services in the Republic of Ireland (ROI) and is the largest employer in the state with over 2,500 workplaces. The policy, which applies to all publicly funded health services, aims to promote awareness and increase consumption of healthier food and drink choices among HSE staff and the visiting public, by highlighting the calorie content of food and drinks provided in HSE facilities (43). Since its introduction in 2015, progress reports suggest inconsistent implementation of the policy across hospitals in Ireland (HSE, personal communication, October, 2018).
With growing evidence that fidelity of implementation is associated with success in achieving intervention outcomes (44–47), a greater understanding of the factors influencing menu labelling implementation in the healthcare setting is required. Research highlights that the process involved in the development and implementation of labelling policies is often context-specific, non-linear and shaped by many different stakeholders and factors (48). The purpose of the current study was to explore the factors that influenced fidelity to a calorie posting policy in Irish public hospitals. To this end, the study objectives were to assess the levels of implementation fidelity to the policy and to identify the perceived factors influencing implementation, and in particular, factors specific to fidelity. This study illustrates methods, applications of theory, and potentially salient factors that may inform future implementation efforts of calorie posting.
Methods
Study Design
The study employed an explanatory sequential mixed methods design (quant → QUAL), with a nested case study for the qualitative component (49). Quantitative data on implementation progress were analysed first, while the qualitative data were collected and analysed second in sequence. The quantitative results provided an initial picture of implementation fidelity across hospitals, while the qualitative analysis refined our understanding by exploring stakeholders' views of the factors that influenced implementation, and more specifically fidelity. In terms of methods, the quantitative and qualitative phases were connected when purposefully selecting cases (hospitals) for the nested case study (50). Finally, the quantitative and qualitative results were integrated during the interpretation of the primary outcome, implementation fidelity as indicated by adherence to the HSE Calorie Posting Policy in hospital staff canteens (see Figure 1 for a diagram of the mixed methods sequential explanatory design).
Figure 1. Diagrammatic representation of the mixed methods sequential explanatory design.
The Consolidated Framework for Implementation Research (CFIR) was used to guide qualitative data collection and analysis. It is a comprehensive, meta-theoretical framework which guides systematic assessment of multi-level influences on implementation (51, 52). The CFIR consists of 39 constructs organised into five major domains: 1. Characteristics of the Intervention (e.g., Evidence Strength & Quality, Relative Advantage), 2. Outer Setting (e.g., Consumer Needs & Resources, External Policies & Incentives), 3. Inner Setting (e.g., Tension for Change, Compatibility), 4. Characteristics of Individuals (e.g., Self-Efficacy, Knowledge & Beliefs), and 5. Process (e.g., Planning, Reflecting & Evaluating). Ethical approval was obtained from the National University of Ireland Galway (ref: 18-Oct-05) and the four participating hospitals. The study has been reported according to published best practises for mixed methods studies (53) (see Supplementary Material 1).
Policy Description
In September 2015, the HSE Calorie Posting Policy was introduced to promote awareness and increase consumption of healthier food and drink choices amongst HSE staff and the visiting public (43). The policy specified the following four conditions must be adhered to in implementing calorie posting:
• Condition 1: Calorie posting is in place for all food and drink items on sale.
• Condition 2: Calorie information is displayed clearly at the “point of choice” for the consumer.
• Condition 3: Calorie information is displayed per standard portion or per meal.
• Condition 4: Information on how many calories an average person needs in a day is prominently displayed to help consumers better understand calorie information.
For more detail on the policy, using the Template for Intervention Description and Replication in Population Health and Policy Interventions Guidelines for Intervention Reporting (54), see the published protocol (55).
Quantitative Phase
The sample consisted of 35 HSE-funded adult acute public hospitals in the ROI with internal catering services who had experience of implementing calorie posting, after the exclusion criteria were applied. Hospitals with external catering services (n = 2) were excluded as these caterers have different levels of resourcing to implement calorie posting which may not be dependent on hospital resources. Specialist hospitals (maternity, paediatric etc., n = 7) were excluded as they were not considered representative of adult acute public hospitals. Hospitals with no experience of calorie posting in staff canteens were also excluded (n = 6). Implementation progress reports were provided by the 35 hospitals (October 2018), and were analysed to assess implementation fidelity and inform case sampling. These reports described whether calories were posted on the breakfast menu only or across the full menu (i.e., adherence to policy condition one). Based on this information, hospitals were categorised as low implementers (i.e., calories on breakfast menu only) and high implementers (i.e., calories across full menu) of the calorie posting policy.
Qualitative Phase
A multiple case study design was used in the qualitative phase. In link with the explanatory sequential design, quantitative data on implementation fidelity at hospitals were used to inform the sampling criteria for cases. Maximum variation sampling was used to select four hospitals with high and low levels of implementation, and variation in terms of geographic location, hospital size, complexity of care provided and hospital type (non-voluntary hospitals–owned and funded by the HSE, and voluntary hospitals–funded but not owned by the HSE). According to Stake (56), a minimum of four cases is recommended; while Creswell (57) suggests no more than four cases to allow individual cases to be adequately explored. Thereafter, a combination of snowball and purposive sampling techniques were used to recruit participants with roles and responsibilities for implementation, including hospital direct (e.g., catering management and staff) and indirect (e.g., hospital senior management, dietitians, health promotion and improvement staff) stakeholders.
Data were collected by the lead investigator (CKer) using multiple methods and included: (1) structured observations; (2) unstructured non-participant observations; (3) in-depth semi-structured interviews. Structured observations, using an observer-rated implementation checklist developed by the study team and refined through initial testing at a pilot hospital, were used to assess implementation of the policy conditions for each menu. Observations were carried out over a 12-h period in staff canteens. Adherence to the four policy conditions for each menu was rated (by CKer) on a scale between 0 and 2 (0 = no, 1 = partially, 2 = yes); thus, generating a total adherence score per menu ranging from 0 (no condition implemented) to 8 (all four conditions fully implemented). The study also used overt, unstructured non-participant observations in staff kitchens/canteens. Field notes were recorded during and/or immediately following the observations. Interviews with direct and indirect stakeholders at each hospital were also conducted. The interview guide was informed by CFIR constructs identified by a recent systematic review of menu labelling determinants (34). Interviews were audio-recorded and transcribed verbatim.
Analysis
The first phase of analysis involved identifying factors influencing implementation, including factors specific to fidelity. In the second phase, data on fidelity were analysed. Findings from both phases were then integrated. The analysis was performed at two levels: within each case and then across the cases (56, 58).
Factors Influencing Implementation
Using NVivo 11 software for data management, framework analysis (59, 60) was performed, with the CFIR as the a priori framework. Thematic analysis was used where data did not fit within the framework (61), and new constructs were added to the codebook. See constructs denoted with an asterisks in Supplementary Material 2. Two researchers (CKer and SMH) independently coded data from one hospital to check for coding consistency and to modify CFIR construct definitions where necessary. For example, the construct Patient Needs & Resources in the Outer Setting domain was modified to Consumer Needs & Resources to reflect that this was a staff facing intervention (see Supplementary Material 2). Once agreed, the lead investigator (CKer) analysed data from the remaining hospitals (see Supplementary Material 2 for final codebook). Using causation coding (61), relationships between constructs were also identified by the lead investigator (CKer) based on interview and observational data for each hospital and were checked independently by another researcher (CR). A consensus-building coding process was adopted throughout the analysis (61).
Implementation Fidelity
To assess fidelity, descriptive statistics were calculated for each hospital using data from structured observations. This included scores per policy condition for each menu and a total score for each menu. An average score across the menus for each hospital was then calculated. Finally, hospitals were categorised into high (average score >4) and low (average score ≤ 4) levels of implementation fidelity by applying cut-off points. Similar to other research (62), these cut-off points were based on minimally accepted practises following discussion and consensus within the research group.
These data were supplemented with other sources of evidence relating to implementation fidelity, not specified in the protocol (55). Information on fidelity was generated during semi-structured interviews and unstructured observations without formal assessment or questioning. Data were analysed thematically to further explore implementation fidelity at each hospital (61).
A triangulation protocol was used to integrate fidelity findings from four sources (i.e. HSE progress reports, structured observations, semi-structured interviews and unstructured observations) (63–65). A “convergence coding matrix” was created with independent findings (as rows) mapped across the data sources (columns). The relationships between data was categorised as: (1) silence (only one data source out of the two being compared contained data on a particular finding), (2) dissonance (conflicting findings in the data), (3) partial agreement (complementarity between data) or (4) agreement (convergence in the data) (65). The triangulation process facilitated the identification of meta-themes that cut across the four sources of fidelity data (63, 64). In line with best practise (63, 64), multiple researchers (CKer, SMH and ET) with combined expertise in qualitative and quantitative methods worked together during triangulation.
Cross-Case Analysis and Integrated Interpretation
Initially, as per the protocol and purposive sampling strategy, the study set out to identify determinants of implementation success by comparing hospitals with high and low levels of fidelity based on the progress reports (55). However, the integrated analysis of fidelity from multiple sources called into question this dichotomy. As a result, the cross-case analysis focused on identifying patterns of fidelity and influencing factors across hospitals. For the cross-case analysis, data were analysed for each hospital first, as outlined above (i.e., examining fidelity per hospital, examining CFIR constructs per hospital). Then findings were compared across hospitals using matrices (displaying results in rows and hospitals in columns). Quantitative and qualitative data were integrated using joint displays to identify meaningful similarities, differences and case-specific experiences (50). In a deviation from the protocol (55), the perspectives of different stakeholders were not compared through formal analysis; however, the type of stakeholder was taken into account when presenting commonly cited influencing factors.
Results
Of the 35 hospitals included in the analysis of fidelity, seven hospitals were categorised as low implementers (i.e., calories on breakfast menu only) and 28 hospitals were high implementers (i.e., calories across full menu) of the calorie posting policy. Characteristics of the four cases are provided in Table 1. The following sections describe factors influencing implementation and within that, factors linked to fidelity and implementation fidelity across hospitals. In an effort to minimise repetition/fragmentation, the results of the cross-case analysis are weaved throughout the results section.
Table 1. Hospital profiles and unstructured observation/interview characteristics.
Factors Influencing Implementation
Factors influencing implementation of the calorie posting policy were coded to constructs across four CFIR domains (as outlined below), with no data coded to the domain which focuses on individual-level constructs (i.e., Characteristics of Individuals) (see Table 2). Most acted simultaneously as barriers and facilitators within and across hospitals. For detailed within and across-case results, including themes and illustrative quotes/observational field notes, see Supplementary Material 3. A matrix of perceived barriers, facilitators and recommendations for future implementation efforts coded to domains and constructs by hospital are provided in Supplementary Materials 4, 5 and 6, respectively. While some factors influencing implementation operated independently, others acted in combination (i.e., with unidirectional and bi-directional relationships). Supplementary Material 7 provides detail of factors which operated in combination, including illustrative quotes/observational field notes. A narrative summary of commonly occurring factors and how they manifested within and across hospitals is presented below.
Table 2. Summary of perceived barriers and facilitators to implementation of calorie posting policy.
Domain: Inner Setting
The internal environment of the hospital was both a barrier and facilitator to implementation across all hospitals. Factors such as the general level of implementation receptivity and readiness presented obstacles and (future) opportunities for implementation. Intervention compatibility with existing work processes exerted a positive and negative influence across all hospitals. Factors which undermined compatibility included changing ingredients or menus, consumers requesting menu variation and required ingredients being unavailable from suppliers. A dietitian manager described this challenge:
… if you're working where you're changing menus regularly… there is extra resource requirement on an ongoing basis to maintain calorie posting in place. It's not like you do it once and that's it… [Dietitian Manager]
Across all hospitals, calorie posting implementation aligned with existing practises or initiatives and thus, made implementation efforts easier. Existing practises or initiatives which enhanced compatibility, noted in interviews and observations, included: undertaking nutritional analysis of patient menus, serving similar meals to patients and staff, posting allergens and implementing workplace health initiatives. In some hospitals, compatibility with work processes led to access to information required for calorie posting implementation. As noted by a senior dietitian:
Some of the food that's available for the patient menus is the same as what's served in the canteen. So a lot of the time while she was processing the patient menus she could just take that information and put it on to the calorie posting for the canteen. [Senior Dietitian]
The lack of perceived importance of calorie posting implementation was a barrier across all hospitals. Stakeholders spoke about calorie posting not being a priority, due to greater importance being placed on patient care, food safety, allergens and other initiatives. In all hospitals, a robust culture of patient-centred care decreased the priority of implementing interventions for staff health and well-being. A dietitian described this challenge:
I don't think there was a huge amount of emphases on calorie posting in the canteen before we had to do it for the patient menus. When it had to be done for the patient menus it was a big priority. Being done for the staff it was take it or leave it. [Senior Dietitian]
The lack of access to knowledge and information about calorie posting and the lack of, or inadequate, training for catering staff was another cited barrier across all hospitals. The reason cited for lack of information was a poorly designed policy with no supporting materials and/or best practise guidelines. In some hospitals, direct stakeholders spoke about overcoming this barrier by accessing information through Google. Across all hospitals, stakeholders highlighted the need for access to knowledge and information to assist with implementation efforts in the future. Recommendations included providing access to structured training for catering staff and clear information on calorie posting implementation. As noted by one member of catering staff:
… there was no supporting information to help us. We were all kind of working at it ourselves through the internet… It was up to yourself to get on with it… we really would have benefited from more information and guidance. [Catering Staff]
Support from leadership was a facilitator across all hospitals. Hospital management were supportive of policy implementation and in some hospitals, provided recognition for implementation efforts and funding for nutrition analysis software. In some hospitals, support from hospital management was considered an important factor for engaging catering staff and management in calorie posting implementation. As one dietitian manager described:
And thankfully hospital management here have been very supportive and they've agreed to fund the licence. We can't do good analysis… without a nutritional analysis software package… [Dietitian Manager]
Stakeholders also highlighted the supportive role of catering management and noted in most hospitals, implementation efforts were led by the assistant catering manager with support from students on placement. Catering management support was driven by a staff-centred or improvement culture within the catering department, improving consumer health, external accreditation/awards and audit/monitoring via researcher visit. As one senior hospital manager described:
… there's a culture within the catering department firstly where they just love to serve staff and they want to give them the nicest and the best food possible… So this was an easy one really because we just knew it was going to build on a good tradition and a good sound belief that they could do better. [Clinical Support Services Director]
The needs and preferences of consumers using the canteen were identified as barriers to implementation. Direct stakeholders discussed the lack of demand for standardised portions, with consumers wanting value for money and, in particular, male consumers having preference for larger portions. Across all hospitals, this led to consumer lack of compliance with standardised portions (e.g., asking for more). As noted by one head chef:
People always want bang for their buck as they say. Some people have smaller appetites and some people have bigger appetites. You can standardise as much as you like within a hot serve but you are going to be dealing with people that will look for more. [Head Chef]
Domain: Process
The process of implementing calorie posting was both a barrier and facilitator across all hospitals. In particular, stakeholders highlighted challenges in engaging catering staff and dietitians in implementation. The lack of or limited engagement of catering staff was attributed, in large part, to catering management not involving staff in implementation. Inadequate staffing levels was another reason cited by stakeholders for lack of or limited engagement of catering staff and dietitians. With competing priorities such as patient services, existing staff had limited time to implement calorie posting. A dietitian manager described these challenges:
Patients are absolutely the priority because we don't have enough staff across the board. It's extremely stretched. Yeah. We don't have resources to take on anything outside. We don't even have the resources to manage the clinical service never mind the health promotion type aspects which are very, very important but we just have to prioritise the patient work. [Dietitian Manager]
Across all hospitals, stakeholders highlighted the positive influence of engaging individuals who were affiliated with an outside entity. Examples of successful engagement with external change agents included obtaining an award from the Irish Heart Foundation for implementing calorie posting, and receiving support from nutrition students on placement in terms of standardising recipes and undertaking nutritional analysis. Stakeholders highlighted that engaging students in implementation compensated for the limited involvement of catering/dietetic management and staff due to time constraints. In addition, the successful engagement of students and the Irish Heart Foundation facilitated direct and indirect stakeholder engagement in hospitals and provided access to information required for implementation. As a catering manager and staff member described:
… my student is doing it because I haven't got the time directly to give it… [Catering Manager]
Oh the Irish Heart Foundation award was very significant… There was huge buy-in then within the catering department. [Catering Staff]
Domain: Outer Setting
The external environment to the hospital facilitated implementation across all hospitals. In particular, external policies and incentives such as (impending) menu labelling legislation, HSE national policy and external accreditation/awards exerted a positive influence on implementation. These external policies and incentives led to hospital/catering management support and engagement, and created an urgency to implement calorie posting. In some hospitals, stakeholders noted that policy requirements to implement calorie posting were more important than consumer demand/interest. As a catering and dietetic manager described:
… external accreditation holds a lot of weight… it makes it easier to negotiate and to kind of keep a door open… particularly at the executive level with the hospital CEO… [Catering Manager]
… it's our national policy the calorie posting in terms of the HSE… Regardless of whether or not consumers want it… its policy so it should be implemented. [Dietitian Manager]
Domain: Intervention Characteristics
Features of the calorie posting policy presented barriers to implementation across all hospitals. In particular, the perceived quality of policy packaging and presentation, including materials and supports available, had a negative influence. Stakeholders described the lack of supporting materials, impractical display options and non-user friendly design as obstacles to implementation. Stakeholders recommended adapting the intervention for future implementation efforts, including introducing a template for calorie posting to help standardise across all hospitals and a more practical calorie display method such as menus electronically displayed with calories. As one catering manager highlighted:
The logistics of trying to keep the poster updated with new menu items and calories…just doesn't work… We've actually applied for funding only last week for a digital menu board… once the information is typed up it will be easier to maintain. [Catering Manager]
Factors Influencing Fidelity
There were barriers and enablers to fidelity to the policy overall and specific dimensions which contributed to an overall pattern of partial adherence evident across hospitals (see Supplementary Material 8).
In some hospitals, inter-related factors cited by direct stakeholders which negatively influenced adherence to the overall policy included lack of internal monitoring of implementation progress over time due to hospital tick-box culture. As highlighted by one head chef:
I think they take on these projects and they look and sound good… They're ticking boxes and that you know… But it's a tick the box. It looks done. But is it really done? No one really checks… I don't think we have fully implemented calorie posting yet. [Executive Head Chef]
In some hospitals, catering staff and management reported the lack of adherence to calorie posting across all food and drink items on sale (policy conditions 1) was due to impractical calorie display methods. As one catering supervisor highlighted:
Now there's still a lot of calories that's not on the big chart. It's difficult to keep it updated with new menu items as need to re-do poster each time. [Catering Supervisor]
Across all hospitals, lack of adherence to standardised portions was attributed, in large part, to lack of consumer demand for standardised portions and expressed preference for larger portions. As one catering manager described:
So like there is a standard portion but like when somebody asks for more they receive more… [Catering Manager]
In some hospitals, indirect stakeholders highlighted lack of training for catering staff was a contributing factor to lack of adherence to standardised recipes and portions. Direct stakeholders in one hospital reported the lack of internal monitoring hindered adherence to standardised recipes; while, direct stakeholders in another hospital noted the presence of internal monitoring enabled greater adherence to standardised portions. As noted by one dietitian manager:
Not giving correct portions is more to do with lack of training, than resistance from the catering people. [Dietitian Manager]
Inaccurate calorie information was largely attributed to lack of adherence to standardised recipes and/or portions across all hospitals. Direct stakeholders in some hospitals highlighted challenges in serving precise standardised quantities of composite dishes such as curries and stews. Student supervision/input was also linked to accuracy of calorie information. Indirect stakeholders in one hospital reported inaccurate calorie information was due to lack of supervision by dietitians; while, direct stakeholders in another hospital noted accurate calorie information was linked to catering management providing student input/guidance. Tied to this, stakeholder level of nutrition knowledge, in particular catering staff and students, was also highlighted by indirect stakeholders as an influential factor in terms of achieving accurate calorie information. Stakeholders described these challenges in obtaining accurate calorie information:
So two ladles of chicken curry could contain different quantities of chicken or veg… So we are not getting an accurate calorie count there… [Executive Head Chef]
I noticed on the posters like some calories straight off as a dietitian I could tell that's not right, that's wrong… because there was no direct supervision from a dietitian I think there are probably a lot of errors in the calculations. [Dietitian Manager]
Implementation Fidelity Across Hospitals
Most comparisons between data sources resulted in “silence”, with the remaining comparisons in “agreement” and some instances of “dissonance” (see Supplementary Material 9). In terms of adherence to the four policy conditions, conflicting findings were noted between data sources: two hospitals were each categorised as having high and low levels of fidelity based on structured observations; while interviews revealed partial adherence across all hospitals. Conflicting findings on adherence to calorie posting across all food and drink items on sale (policy conditions 1) were also noted between the HSE progress reports and that of structured observations and interviews.
Both structured observations and interviews showed a consistent pattern of low adherence to calorie posting across all food and drink items on sale (policy conditions 1) and calorie information displayed per standard portion or per meal (policy condition 3) across all hospitals. There was variability in adherence to the remaining policy conditions (2 and 4) across hospitals.
Overall, the integrated analysis of fidelity findings from four sources showed a pattern of partial adherence to the calorie posting policy across all four hospitals, with evidence of variation in specific fidelity dimensions (see Table 3).
Table 3. Summary of indicators of fidelity according to data sources.
Discussion
This mixed methods study explored factors influencing fidelity to a calorie posting policy in Irish acute public hospitals. Integrated findings from multiple data sources revealed different levels of fidelity depending on the aspect of the policy under consideration and the source of data. There was an overall pattern of partial adherence across hospitals. Across all hospitals, there was a consistent pattern of low adherence to calorie posting across all menu items on sale, low adherence to calorie information displayed per standard portion or per meal, low adherence to standardised recipes/portions, and inaccurate calorie information. Factors influencing fidelity operated independently and in combination, and were evident across four domains: the internal and external environment of the hospital, the process involved in implementation and features of the calorie posting policy. For example, external policies and incentives (e.g., national policy, monitoring) and features of the calorie posting policy (e.g., availability of supporting materials) influenced hospital receptivity and readiness to implement (e.g., leadership support, access to knowledge and information, perceived importance of calorie posting implementation) and subsequently, the process involved in implementation (e.g., engaging relevant stakeholders). Findings highlight the complex and dynamic influences at play during implementation.
Recent research highlights a nuanced relationship between direct (i.e., observer report) and indirect (i.e., self-report) measures of fidelity, where agreement between both measures is stronger for some intervention components than others (66–68). The current study found good agreement between direct and indirect measures across most dimensions of fidelity; however, there was some instances of disagreement. This included conflicting findings between HSE progress reports (October 2018) and that of structured observations and interviews (September/October 2019), which may reflect differences in fidelity assessment across time (68). Disagreement between structured observations and interviews was also evident, where stakeholders from two hospitals rated themselves lower than the observer as regards adherence to the four policy conditions. This finding is in contrast with previous research which show individuals report higher fidelity than observers (69–73). A possible explanation for this may include differences in conceptualisation of high and low fidelity by researchers and implementers. For example, cut-off points were decided arbitrarily by the research team and may have benefited from stakeholder engagement in this process.
Analysis of fidelity using direct and indirect measures highlighted core intervention components not specified in the calorie posting policy, including adherence to standardised recipes/portions and accuracy of calorie information. This may reflect the low level of specificity with which essential components (i.e., the four policy conditions) were defined (74, 75) or the lack of interrogation of the intervention logic for calorie posting in the literature to enable specification of intervention components (76, 77). Nevertheless, research shows clear specificity of what the intervention or policy entails is necessary to ensure effective implementation (44, 78). Hawes (76, 79) also argues that intervention components should be clearly identified as relating to form (i.e., variable aspect of the intervention–calorie display method) and function (i.e., fixed aspect of the intervention–accurate calorie information per standard portion or per meal). So there is standardisation by function rather than form across sites, which allows for adaptation to context while maintaining fidelity (79).
Similar to research on other innovations in the health care setting (80), the lack of access to information and training was an obstacle to implementation. Studies show training of key staff is a predictor of implementation success (81–84). Although expressed as a facilitator to implementation, accessing information via the web may be an unreliable source of information and thus, may have contributed to inaccurate calorie information. Study participants were unaware of a “Calorie Posting Toolkit” which had been designed to assist hospitals when implementing the policy (85); this may have contributed to partial adherence across hospitals. According to a recent study (86), implementation of evidence-based interventions in real-world settings become a futile effort when effective strategies to help implementation are not used.
There is evidence to suggest that organisations with cultures that are more supportive of employee health and well-being are more effective in implementing evidence-based practises (87). In the current study a lower priority was placed on implementing interventions such as calorie posting designed for staff health and well-being due to inadequate staffing levels and a culture which prioritised patient-centred care. While this was a common barrier to implementation across all hospitals, it was specifically linked to lack of adherence to the overall policy in some hospitals. These findings are similar to previous research which indicate that health care resources are under pressure and subsequently, patient-based tasks are the priority, with little or no time available for staff to implement new or existing evidence-based practises (80). Furthermore, other research has shown that lack of perceived importance due to limited resource availability is associated with low levels of implementation success (88–91).
In response to staff shortages and time restraints, students on placement were engaged in calorie posting implementation across all hospitals. Although the supportive role of external change agents is well documented (92), the current study identified issues with lack of appropriate student supervision and their varying levels of nutrition knowledge. According to a recent study (93), intervention responsibility and accountability need to be considered when engaging change agents. The authors of that study and others highlight the potential harm of over-reliance on these individuals and its impact on sustainable intervention practise (51, 93, 94).
The needs and preferences of staff utilising the canteen (i.e., consumers) was perceived to undermine adherence to standardised portions and subsequently, result in inaccurate calorie information. As noted in a recent systematic review of menu labelling determinants (34), these findings may reflect the lack of consumer education about how to utilise menu labelling effectively. The CFIR places the service user (i.e., patients/consumers) under the outer setting domain, suggesting their peripheral role in the implementation process (95). In the current study, the addition of a new construct on consumer needs and preferences to the inner setting domain demonstrated the central role of consumers in the implementation process. This is in line with recommendations in the implementation science literature (95–97), were greater consideration of the service user role in implementation is required.
In general, policies and incentives external to hospitals exerted a positive influence by garnering hospital/catering management support and generating an urgency to implement calorie posting. These findings reflect key aspects of institutional theory which suggest that organisations change as an adaptive response to coercion, or to strong pressures to comply with rules, regulations, and mandates (98, 99). Research also shows organisations will implement innovations to comply with accrediting bodies (100). While it is recognised that national policy/regulation is required to advance public health (17, 48, 101), the risk of tokenism in the form of superficial implementation needs to be averted (102). In the current study, a hospital tick-box culture led to lack of monitoring implementation which hindered policy adherence. These finding highlight the need for effective mechanisms to be put in place to ensure rigorous monitoring alongside policy (31, 103).
Implications and Recommendations for Policy, Practice, and Research
The study findings point to the need for multi-level, multi-component strategies to maximise fidelity. Efforts to promote fidelity should begin with the initial process of policy development. “Designing for fidelity” requires adequate documentation of intervention components and a description of activities so that they can be replicated (104). Ensuring the availability of supporting materials, such as a calorie posting toolkit, and the provision of training opportunities are necessary to equip stakeholders with the required knowledge and information (86, 105). For these resources to be properly utilised, stakeholders involved in implementation need to know how to access and interpret them (80). Supporting implementation also requires consideration of human resources and its impact on time and capacity to implement new innovations (106, 107). In the face of competing demands, there is a need to prioritise staff health and create a culture that promotes and supports a healthy workforce (108).
To maximise fidelity and reduce “intervention drift”, implementation needs to be adequately and continuously monitored via internal and external sources (68, 77, 104, 109, 110). A feedback loop is also required along with encouragement, reward or recognition for implementation efforts (104). Both monitoring and the provision of feedback are considered key to maximising stakeholder engagement and contributing to effective implementation (104). The Plan-Do-Study-Act cycle is a commonly used process improvement strategy in health care settings, which allows organisations to initiate, evaluate and refine over a short time period (111–113). Furthermore, given the volunteer nature associated with student engagement, training and supervision must also be a priority to ensure a high degree of fidelity (114).
Few implementation models/frameworks, including the CFIR (51), recognise that different factors may influence implementation at different points in the implementation process (115). Future research should determine the factors relevant to different phases of implementation, so as to help stakeholders to anticipate and address factors in sequence or in tandem for effective implementation (34). A newly developed method called Coincidence Analysis (116), offers the potential to identify combinations of conditions that are minimally necessary or sufficient for effective implementation. Furthermore, the association between implementation strategies and shifts in fidelity to intervention core components could be examined in future studies (68).
Limitations and Strengths
While the sample of hospitals was diverse in terms of geographic region, size and type, they were all public hospitals with internal catering services. Thus, findings may not be transferable to private hospitals and those with external catering services. In an effort to help minimise the effects of recall and response bias, data from multiple sources and types were used (58, 117). Despite the recognised benefits of using multiple measures of fidelity (77, 118), there were challenges to integrating fidelity data due to different levels or aspects of fidelity evident in different sources. A diffractive approach, whereby patterns of difference and entanglement are identified, may have offered another means to capture the complexity and messiness of fidelity findings (119). While multiple perspectives from different stakeholders were obtained, the study did not consider the consumer perspective. Finally, the study analysis goes beyond simply listing influential factors by highlighting interactions and their effect on adherence to a calorie posting policy.
Conclusion
This study found partial adherence to a calorie posting policy across hospitals, where influential factors were multiple, and operated independently and in combination. Factors were related to the hospital internal and external environment, features of the calorie posting policy and the process involved in implementation. The importance of multiple measures of fidelity to generate accurate and more comprehensive fidelity findings was also evident. Findings point to the need for multi-level, multicomponent strategies to maximise fidelity. Future research should assess the relative importance of calorie posting determinants and the association between implementation strategies and shifts in fidelity to intervention core components.
Data Availability Statement
The data analysed during the current study are not publicly available in order to protect the privacy and confidentiality of all study hospitals and participants.
Ethics Statement
Ethical approval was obtained from the National University of Ireland Galway (ref: 18-Oct-05) and the four participating hospitals. Verbal consent was obtained from hospital gatekeepers to conduct the study. Written informed consent was obtained from study participants in each hospital.
Author Contributions
CKer was the lead investigator and drafted the manuscript. CKer, CKel, CHo, ET, CHa, FG, IP, and SMH contributed to the concept and design of the study. CKer collected data and led the analysis with input from CR, SMH, CKel, and ET. All authors were involved in the interpretation of results, and critically reviewed and approved the final manuscript.
Funding
This research was funded by the Health Research Board (HRB) SPHeRE/2013/1 and the Health Service Executive (HSE) in Ireland. SMH is funded through the HRB Research Leader Award (RL-2020-004). The HRB had no role in the planning, conduct, or interpretation of the study. The HSE were involved in the initial study design but had no role in data collection, analysis of data, interpretation of data or in writing the manuscript. The content of this manuscript is the responsibility of the authors and does not necessarily reflect the views of the funding bodies.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Acknowledgments
The authors would like to acknowledge the contributions of the advisory group established by the Healthy Eating Active Living Policy Priority Programme, as part of Health & Wellbeing, Strategic Planning and Transformation of the HSE in Ireland, for their input to the design of this study. The authors also wish to thank all hospitals and stakeholders who participated in this study.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2021.707668/full#supplementary-material
References
1. World Health Organisation (WHO). Obesity and Overweight. Available online at: https://www.who.int/news-room/fact-sheets/detail/obesity-and-overweight (accessed April 8, 2021).
2. Vandenbroeck IP, Goossens J, Clemens M. Foresight Tackling Obesities: Future Choices—Obesity System Atlas. Available online at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/295153/07-1177-obesity-system-atlas.pdf (accessed July 23, 2021).
3. Bes-Rastrollo M, Basterra-Gortari FJ, Sanchez-Villegas A, Marti A, Martínez JA, Martínez-González MA, et al. Prospective study of eating away-from-home meals and weight gain in a Mediterranean population: the SUN (Seguimiento Universidad de Navarra) cohort. Public Health Nutr. (2010) 13:1356–63. doi: 10.1017/S1368980009992783
4. Bhutani S, Schoeller DA, Walsh MC, McWilliams C. Frequency of eating out at both fast-food and sit-down restaurants was associated with high body mass index in non-large metropolitan communities in midwest. AJHP. (2018) 32:75–83. doi: 10.1177/0890117116660772
5. Bezerra IN, Curioni C, Sichieri R. Association between eating out of home and body weight. Nutr Rev. (2012) 70:65–79. doi: 10.1111/j.1753-4887.2011.00459.x
6. Nago ES, Lachat CK, Dossa RAM, Kolsteren PW. Association of out-of-home eating with anthropometric changes: a systematic review of prospective studies. Crit Rev Food Sci Nutr. (2014) 54:1103–16. doi: 10.1080/10408398.2011.627095
7. Todd JE. Changes in consumption of food away from home and intakes of energy and other nutrients among US working-age adults, 2005–2014. Public Health Nutr. (2017) 20:3238–46. doi: 10.1017/S1368980017002403
8. Smith LP, Ng SW, Popkin BM. Trends in US home food preparation and consumption: analysis of national nutrition surveys and time use studies from 1965–1966 to 2007–2008. Nutr J. (2013) 12:45. doi: 10.1186/1475-2891-12-45
9. Mills S, Adams J, Wrieden W, White M, Brown H. Sociodemographic characteristics and frequency of consuming home-cooked meals and meals from out-of-home sources: cross-sectional analysis of a population-based cohort study. Public Health Nutr. (2018) 21:2255–66. doi: 10.1017/S1368980018000812
10. Robinson E, Jones A, Whitelock V, Mead BR, Haynes A. (Over) eating out at major UK restaurant chains: observational study of energy content of main meals. BMJ. (2018) 363:k4982. doi: 10.1136/bmj.k4982
11. Theis DRZ, Adams J. Differences in energy and nutritional content of menu items served by popular UK chain restaurants with versus without voluntary menu labelling: A cross-sectional study. PLoS ONE. (2019) 14:e0222773. doi: 10.1371/journal.pone.0222773
12. Muc M, Jones A, Roberts C, Sheen F, Haynes A, Robinson E, et al. Bit or a lot on the side? Observational study of the energy content of starters, sides and desserts in major UK restaurant chains. BMJ Open. (2019) 9:e029679. doi: 10.1136/bmjopen-2019-029679
13. Roberts SB, Das SK, Suen VMM, Pihlajamäki J, Kuriyan R, Steiner-Asiedu M, et al. Measured energy content of frequently purchased restaurant meals: multi-country cross sectional study. BMJ. (2018) 363:k4864. doi: 10.1136/bmj.k4864
14. Block JP, Condon SK, Kleinman K, Mullen J, Linakis S, Rifas-Shiman S, et al. Consumers' estimation of calorie content at fast food restaurants: cross sectional observational study. BMJ. (2013) 346:f2907. doi: 10.1136/bmj.f2907
15. Burton S, Tangari AH, Howlett E, Turri AM. How the perceived healthfulness of restaurant menu items influences sodium and calorie misperceptions: Implications for nutrition disclosures in chain restaurants. J Consum Aff. (2014) 48:62–95. doi: 10.1111/joca.12015
16. Robinson E, Marty L, Jones A, White M, Smith R, Adams J. Will calorie labels for food and drink served outside the home improve public health? BMJ. (2021) 372:n40. doi: 10.1136/bmj.n40
17. Mozaffarian D, Angell SY, Lang T, Rivera JA. Role of government policy in nutrition—barriers to and opportunities for healthier eating. BMJ. (2018) 361:k2426. doi: 10.1136/bmj.k2426
18. Marteau TM, White M, Rutter H, Petticrew M, Mytton OT, McGowan JG, et al. Increasing healthy life expectancy equitably in England by 5 years by 2035: could it be achieved? Lancet. (2019) 393:2571–3. doi: 10.1016/S0140-6736(19)31510-7
19. Food & Drug Administration (FDA). Menu and Vending Machines Labelling Requirements. Available online at: https://www.fda.gov/food/food-labeling-nutrition/overview-fda-labeling-requirements-restaurants-similar-retail-food-establishments-and-vending (accessed April 8, 2021).
20. NSW Food Authority. Kj Labelling Nutrition Information. Available online at: http://www.foodauthority.nsw.gov.au/ip/legislation/proposals-and-policy/kJ-labelling-nutrition-information (accessed April 8, 2021).
21. Ontario Ministry of Health and Long-Term Care. New Menu Labelling Regulation. Available online at: http://health.gov.on.ca/en/news/bulletin/2016/hb_20160927.aspx (accessed April 8, 2021).
22. Food Safety Authority of Ireland (FSAI). Putting Calories on Menus in Ireland - Technical Guidance for Food Businesses. Dublin: FSAI. (2012).
23. Department of Health and Social Care. Consultation on mandating calorie labelling in the out-of home sector. Available online at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/751529/consultation-on-calorie-labelling-outside-of-the-home.pdf (accessed April 8, 2021).
24. Health Service Executive (HSE). Guidance for Calorie Posting Implementation. Dublin: HSE (2015).
25. Vanderlee L, Hammond D. Does nutrition information on menus impact food choice? Comparisons across two hospital cafeterias. Public Health Nutr. (2014) 17:1393–402. doi: 10.1017/S136898001300164X
26. Vyth EL, Steenhuis IH, Heymans MW, Roodenburg AJ, Brug J, Seidell JC. Influence of placement of a nutrition logo on cafeteria menu items on lunchtime food choices at Dutch work sites. J Am Diet Assoc. (2011) 111:131–6. doi: 10.1016/j.jada.2010.10.003
27. Crockett R, King S, Marteau T, Prevost A, Bignardi G, Roberts N, et al. Nutritional labelling for healthier food or non-alcoholic drink purchasing and consumption. Cochrane Database Syst Rev. (2018) 2:CD009315. doi: 10.1002/14651858.CD009315.pub2
28. Shangguan S, Afshin A, Shulkin M, Ma W, Marsden D, Smith J, et al. A meta-analysis of food labelling effects on consumer diet behaviours and industry practises. Am J Prev Med. (2019) 56:300–14. doi: 10.1016/j.amepre.2018.09.024
29. Fitzgerald S, Gilgan L, McCarthy M, Perry IJ, Geaney F. An evaluation and exploration of Irish food-service businesses' uptake of and attitudes towards a voluntary government-led menu energy (calorie) labelling initiative. Public Health Nutr. (2018) 21:3178–91. doi: 10.1017/S1368980018001969
30. Robinson E, Burton S, Gough T, Jones A, Haynes A. Point of choice kilocalorie labelling in the UK eating out of home sector: a descriptive study of major chains. BMC Public Health. (2019) 19:649. doi: 10.1186/s12889-019-7017-5
31. Huang Y, Pomeranz JL, Cash SB. Effective national menu labelling requires accuracy and enforcement. J Acad Nutr Diet. (2018) 118:989–93. doi: 10.1016/j.jand.2018.03.001
32. Urban LE, McCrory MA, Dallal GE, Das SK, Saltzman E, Weber JL, et al. Accuracy of stated energy contents of restaurant foods. JAMA. (2011) 306:287–93. doi: 10.1001/jama.2011.993
33. Feldman C, Murray D, Chavarria S, Zhao H. Menu label accuracy at a university's foodservices. An exploratory recipe nutrition analysis. Appetite. (2015) 92:24–8. doi: 10.1016/j.appet.2015.05.001
34. Kerins C, McHugh S, McSharry J, Reardon CM, Hayes C, Perry IJ, et al. Barriers and facilitators to implementation of menu labelling interventions from a food service industry perspective: a mixed methods systematic review. ?Int J Behav Nutr Phys Act. (2020) 17:48. doi: 10.1186/s12966-020-00948-1
35. Vasiljevic M, Cartwright E, Pilling M, Lee M-M, Bignardi G, Pechey R, et al. Impact of calorie labelling in worksite cafeterias: a stepped wedge randomised controlled pilot trial. Int J Behav Nutr Phys Act. (2018) 15:41. doi: 10.1186/s12966-018-0671-7
36. Vyth EL, Van Der Meer EW, Seidell JC, Steenhuis IH A. nutrition labelling intervention in worksite cafeterias: an implementation evaluation across two large catering companies in the Netherlands. Health Promot Int. (2012) 27:230–7. doi: 10.1093/heapro/dar034
37. Artazcoz L, Cortès I, Benavides FG, Escribà-Agüir V, Bartoll X, Vargas H, et al. Long working hours and health in Europe: Gender and welfare state differences in a context of economic crisis. Health Place. (2016) 40:161–8. doi: 10.1016/j.healthplace.2016.06.004
38. Cygan-Rehm K, Wunder C. Do working hours affect health? Evidence from statutory workweek regulations in Germany. Labour Econ. (2018) 53:162–71. doi: 10.1016/j.labeco.2018.05.003
39. Johnson JV, Lipscomb J. Long working hours, occupational health and the changing nature of work organisation. Am J Ind Med. (2006) 49:921–9. doi: 10.1002/ajim.20383
40. Blanck HM, Yaroch AL, Atienza AA Yi SL, Zhang J, Mâsse LC. Factors influencing lunchtime food choices among working Americans. Health Educ Behav. (2009) 36:289–301. doi: 10.1177/1090198107303308
41. Roos E, Sarlio-Lähteenkorva S, Lallukka T. Having lunch at a staff canteen is associated with recommended food habits. Public Health Nutr. (2004) 7:53–61. doi: 10.1079/PHN2003511
42. Rothstein MA. Promoting public health in health care facilities. Am J Public Health. (2014) 104:965–7. 10.2105/AJPH.2014.301885 doi: 10.2105/AJPH.2014.301885
44. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S, et al. conceptual framework for implementation fidelity. Implement Sci. (2007) 2:40. doi: 10.1186/1748-5908-2-40
45. Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. (2008) 41:327–50. doi: 10.1007/s10464-008-9165-0
46. Eames C, Daley D, Hutchings J, Hughes JC, Jones K, Martin P, et al. The Leader Observation Tool: a process skills treatment fidelity measure for the Incredible Years parenting programme. Child Care Health Dev. (2008) 34:391–400. doi: 10.1111/j.1365-2214.2008.00828.x
47. Johnson-Kozlow M, Hovell MF, Rovniak LS, Sirikulvadhana L, Wahlgren DR, Zakarian JM. Fidelity issues in secondhand smoking interventions for children. Nicotine Tob Res. (2008) 10:1677–90. doi: 10.1080/14622200802443429
48. World Cancer Research Fund International (WCRF International). Building Momentum: Lessons on Implementing a Robust Front-of-Pack Food Label. London: WCRF International (2019).
49. Guetterman TC, Fetters MD. Two Methodological approaches to the integration of mixed methods and case study designs: a systematic review. Am Behav Sci. (2018) 62:900–18. doi: 10.1177/0002764218772641
50. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practises. Health Serv Res. (2013) 48:2134–56. doi: 10.1111/1475-6773.12117
51. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practise: a consolidated framework for advancing implementation science. Implement Sci. (2009) 4:50. doi: 10.1186/1748-5908-4-50
52. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. (2015) 10:53. doi: 10.1186/s13012-015-0242-0
53. O'Cathain A, Murphy E, Nicholl J. The quality of mixed methods studies in health services research. J Health Serv Res Policy. (2008) 13:92–8. doi: 10.1258/jhsrp.2007.007074
54. Campbell M, Katikireddi SV, Hoffmann T, Armstrong R, Waters E, Craig P. TIDieR-PHP: a reporting guideline for population health and policy interventions. BMJ. (2018) 361:k1079. doi: 10.1136/bmj.k1079
55. Kerins C, Houghton C, McHugh S, Geaney F, Toomey E, Hayes C, et al. Implementation of a calorie menu labelling policy in public hospitals: study protocol for a multiple case study. Int J Qual Methods. (2019) 18:1–10. doi: 10.1177/1609406919878339
57. Creswell JW. Qualitative inquiry and research design: Choosing among the five traditions. Thousand Oaks, CA: Sage. (1998).
58. Yin RK. Case study research and applications: design and methods. 6th ed Los Angeles: Sage Publications. (2018).
59. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. (2013) 13:117. doi: 10.1186/1471-2288-13-117
60. Smith J, Firth J. Qualitative data analysis: the framework approach. Nurse Res. (2011) 18:52–62. doi: 10.7748/nr2011.01.18.2.52.c8284
61. Saldana J. The coding manual for qualitative researchers. 4th ed London: Sage Publications. (2021).
62. Swindle T, Johnson SL, Davenport K, Whiteside-Mansell L, Thirunavukarasu T, Sadasavin G, et al. A mixed-methods exploration of barriers and facilitators to evidence-based practises for obesity prevention in head start. J Nutr Educ Behav. (2019) 51:1067–79.e1. doi: 10.1016/j.jneb.2019.06.019
63. O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. (2010) 341:c4587. doi: 10.1136/bmj.c4587
64. Farmer T, Robinson K, Elliott SJ, Eyles J. Developing and implementing a triangulation protocol for qualitative health research. Qual Health Res. (2006) 16:377–94. doi: 10.1177/1049732305285708
65. Tonkin-Crine S, Anthierens S, Hood K, Yardley L, Cals JWL, Francis NA, et al. Discrepancies between qualitative and quantitative evaluation of randomised controlled trial results: achieving clarity through mixed methods triangulation. Implement Sci. (2016) 11:66. doi: 10.1186/s13012-016-0436-0
66. Ward AM, Regan J, Chorpita BF, Starace N, Rodriguez A, Okamura K, et al. Tracking evidence based practise with youth: validity of the MATCH and standard manual consultation records. J Clin Child Adolesc Psychol. (2013) 42:44–55. doi: 10.1080/15374416.2012.700505
67. Hogue A, Dauber S, Lichvar E, Bobek M, Henderson CE. Validity of therapist self-report ratings of fidelity to evidence-based practises for adolescent behaviour problems: correspondence between therapists and observers. Adm Policy Ment Health. (2015) 42:229–43. doi: 10.1007/s10488-014-0548-2
68. Swindle T, Selig JP, Rutledge JM, Whiteside-Mansell L, Curran G. Fidelity monitoring in complex interventions: a case study of the WISE intervention. Arch Public Health. (2018) 76:53. doi: 10.1186/s13690-018-0292-2
69. Martino S, Ball S, Nich C, Frankforter TL, Carroll KM. Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychother Res. (2009) 19:181–93. doi: 10.1080/10503300802688460
70. Toomey E, Matthews J, Hurley DA. Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain. BMJ Open. (2017) 7:e015452. doi: 10.1136/bmjopen-2016-015452
71. Hardeman W, Michie S, Fanshawe T, Prevost AT, McLoughlin K, Kinmonth AL. Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychol Health. (2008) 23:11–24. doi: 10.1080/08870440701615948
72. Breitenstein SM, Fogg L, Garvey C, Hill C, Resnick B, Gross D. Measuring implementation fidelity in a community-based parenting intervention. Nurs Res. (2010) 59:158–65. doi: 10.1097/NNR.0b013e3181dbb2e2
73. Carroll K, Nich C, Rounsaville B. Utility of therapist session checklists to monitor delivery of coping skills treatment for cocaine abusers. Psychother Res. (1998) 8:307–20. doi: 10.1093/ptr/8.3.307
74. Haynes A, Brennan S, Redman S, Williamson A, Gallego G, Butow P, et al. Figuring out fidelity: a worked example of the methods used to identify, critique and revise the essential elements of a contextualised intervention in health policy agencies. Implement Sci. (2016) 11:23. doi: 10.1186/s13012-016-0378-6
75. Stains M, Vickrey T. Fidelity of implementation: an overlooked yet critical construct to establish effectiveness of evidence-based instructional practises. CBE Life Sci Educ. (2017) 16. doi: 10.1187/cbe.16-03-0113
76. Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ (Clinical research ed). (2004) 328:1561–3. doi: 10.1136/bmj.328.7455.1561
77. Toomey E, Hardeman W, Hankonen N, Byrne M, McSharry J, Matvienko-Sikar K, et al. Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions. Health Psychol Behav Med. (2020) 8:132–51. doi: 10.1080/21642850.2020.1738935
78. Dusenbury L, Brannigan R, Falco M, Hansen WB A. review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. (2003) 18:237–56. doi: 10.1093/her/18.2.237
79. Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health. (2015) 36:307–23. doi: 10.1146/annurev-publhealth-031912-114421
80. Williams B, Perillo S, Brown T. What are the factors of organisational culture in health care settings that act as barriers to the implementation of evidence-based practise? A scoping review. Nurse Educ Today. (2015) 35:e34–41. doi: 10.1016/j.nedt.2014.11.012
81. Hodge LM, Turner KM. Sustained Implementation of Evidence-based Programs in Disadvantaged Communities: A Conceptual Framework of Supporting Factors. Am J Community Psychol. (2016) 58:192–210. doi: 10.1002/ajcp.12082
82. Han SS, Weiss B. Sustainability of teacher implementation of school-based mental health programs. J Abnorm Child Psychol. (2005) 33:665–79. doi: 10.1007/s10802-005-7646-2
83. Forman J, Harrod M, Robinson C, Annis-Emeott A, Ott J, Saffar D, et al. First things first: foundational requirements for a medical home in an academic medical centre. J Gen Intern Med. (2014) 29 Suppl 2:S640–8. doi: 10.1007/s11606-013-2674-z
84. Green CA, McCarty D, Mertens J, Lynch FL, Hilde A, Firemark A, et al. A qualitative study of the adoption of buprenorphine for opioid addiction treatment. J Subst Abuse Treat. (2014) 46:390–401. doi: 10.1016/j.jsat.2013.09.002
85. Health Service Executive (HSE). Calorie Posting Toolkit. Available online at: https://www.hse.ie/eng/about/who/healthwellbeing/our-priority-programmes/heal/calorie-posting/ (accessed April 6, 2021).
86. Thoele K, Ferren M, Moffat L, Keen A, Newhouse R. Development and use of a toolkit to facilitate implementation of an evidence-based intervention: a descriptive case study. Implement Sci Commun. (2020):86. doi: 10.1186/s43058-020-00081-x
87. Aarons GA, Moullin JC, Ehrhart MG. The role of organisational processes in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: Translating science to practise. 2nd ed. New York: Oxford University Press (2018). p. 121–42. doi: 10.1093/oso/9780190683214.003.0008
88. Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. (2013) 8:51. doi: 10.1186/1748-5908-8-51
89. Damschroder LJ, Goodrich DE, Robinson CH, Fletcher CE, Lowery JC A. systematic exploration of differences in contextual factors related to implementing the MOVE! weight management program in VA: a mixed methods study. BMC Health Serv Res. (2011) 11:248. doi: 10.1186/1472-6963-11-248
90. Weiner BJ, Haynes-Maslow L, Kahwati LC, Kinsinger LS, Campbell MK. Implementing the MOVE! weight-management program in the Veterans Health Administration, 2007-2010: a qualitative study. Prev Chronic Dis. (2012) 9:E16. doi: 10.5888/pcd9.110127
91. Klein KJ, Conn AB, Sorra JS. Implementing computerised technology: an organisational analysis. J Appl Psychol. (2001) 86:811–24. doi: 10.1037/0021-9010.86.5.811
92. Alagoz E, Chih MY, Hitchcock M, Brown R, Quanbeck A. The use of external change agents to promote quality improvement and organisational change in healthcare organisations: a systematic review. BMC Health Serv Res. (2018) 18:42. doi: 10.1186/s12913-018-2856-9
93. Evans R, Murphy S, Scourfield J. Implementation of a school-based social and emotional learning intervention: understanding diffusion processes within complex systems. Prev Sci. (2015) 16:754–64. doi: 10.1007/s11121-015-0552-0
94. Rycroft-Malone J, Kitson A, Harvey G, McCormack B, Seers K, Titchen A, et al. Ingredients for change: revisiting a conceptual framework. BMJ Qual Saf. (2002) 11:174. doi: 10.1136/qhc.11.2.174
95. Varsi C, Ekstedt M. Using the Consolidated Framework for Implementation Research to Identify Barriers and Facilitators for the Implementation of an Internet-Based Patient-Provider Communication Service in Five Settings: A Qualitative Study. J Med Internet Res. (2015) 17:e262. doi: 10.2196/jmir.5091
96. Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res. (2019) 19:189. doi: 10.1186/s12913-019-4015-3
97. Nevedal AL, Reardon CM, Jackson GL, Cutrona SL, White B, Gifford AL, et al. Implementation and sustainment of diverse practises in a large integrated health system: a mixed methods study. Implement Sci Commun. (2020) 1:61. doi: 10.1186/s43058-020-00053-1
98. Birken SA, Bunger AC, Powell BJ, Turner K, Clary AS, Klaman SL, et al. Organisational theory for dissemination and implementation research. Implement Sci. (2017) 12:62. doi: 10.1186/s13012-017-0592-x
99. Hannan MT, Freeman J. The population ecology of organisations. Am J Sociol. (1977) 82:929–64. doi: 10.1086/226424
100. Birken SA, Mayer DK, Weiner BJ. Survivorship care plans: prevalence and barriers to use. J Cancer Educ. (2013) 28:290–6. doi: 10.1007/s13187-013-0469-x
101. Swinburn BA, Kraak VI, Allender S, Atkins VJ, Baker PI, Bogard JR, et al. The global syndemic of obesity, undernutrition, and climate change: The Lancet Commission report. Lancet. (2019) 393:791–846. doi: 10.1016/S0140-6736(18)32822-8
102. Powell WW, DiMaggio PJ. The new institutionalism in organisational analysis. Chicago: University of Chicago Press. (2012).
103. Raine KD, Ferdinands AR, Atkey K, Hobin E, Jeffery B, Nykiforuk CI, et al. Policy recommendations for front-of-package, shelf, and menu labelling in Canada: Moving towards consensus. Can J Public Health. (2017) 108:409–13. doi: 10.17269/CJPH.108.6076
104. Allen JD, Shelton RC, Emmons KM, Linnan LA. Fidelity and its relationship to implementation effectiveness, adaptation, and dissemination. In: Brownson RC, Colditz GA, Proctor EK, editors. Dissemination and implementation research in health: Translating science to practise. 2nd ed. New York: Oxford University Press (2018). p. 267–84. doi: 10.1093/oso/9780190683214.003.0016
105. Liang S, Kegler MC, Cotter M, Emily P, Beasley D, Hermstad A, et al. Integrating evidence-based practises for increasing cancer screenings in safety net health systems: a multiple case study using the Consolidated Framework for Implementation Research. Implement Sci. (2016) 11:109. doi: 10.1186/s13012-016-0477-4
106. Fitzgerald L, Ferlie E, Wood M, Hawkins C. Interlocking interactions, the diffusion of innovations in health care. Hum Relat. (2002) 55:1429–49. doi: 10.1177/001872602128782213
107. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organisations: systematic review and recommendations. Milbank Q. (2004) 82:581–629. doi: 10.1111/j.0887-378X.2004.00325.x
108. Quirk H, Crank H, Carter A, Leahy H, Copeland RJ. Barriers and facilitators to implementing workplace health and wellbeing services in the NHS from the perspective of senior leaders and wellbeing practitioners: a qualitative study. BMC Public Health. (2018) 18:1362. doi: 10.1186/s12889-018-6283-y
109. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behaviour change studies: best practises and recommendations from the NIH Behaviour Change Consortium. Health Psychol. (2004) 23:443. doi: 10.1037/0278-6133.23.5.443
110. Bova C, Jaffarian C, Crawford S, Quintos JB, Lee M, Sullivan-Bolyai S. Intervention fidelity: monitoring drift, providing feedback, and assessing the control condition. Nurs Res. (2017) 66:54–9. doi: 10.1097/NNR.0000000000000194
111. Schneider A. How quality improvement in health care can help to achieve the Millennium Development Goals. Bull World Health Organ. (2006) 84:259. doi: 10.2471/BLT.06.031385
112. Evans AC, Rieckmann T, Fitzgerald MM, Gustafson DH. Teaching the NIATx model of process improvement as an evidence-based process. JTADD. (2008) 6:21–37. doi: 10.1080/15332700802127912
113. Ragsdale MA, Mueller J. Plan, do, study, act model to improve an orientation program. J Nurs Care Qual. (2005) 20:268–72. doi: 10.1097/00001786-200507000-00013
114. Barwick M, Barac R, Zlotkin S. An examination of exclusive breastfeeding implementation in Ethiopia and Mali: factors influencing change. Ontario: CARE Canada; Save the Children Canada; Plan International Canada. (2015).
115. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practise implementation in public service sectors. Adm Policy Ment Health. (2011) 38:4–23. doi: 10.1007/s10488-010-0327-7
116. Whitaker RG, Sperber N, Baumgartner M, Thiem A, Cragun D, Damschroder L, et al. Coincidence analysis: a new method for causal inference in implementation science. Implement Sci. (2020) 15:108. doi: 10.1186/s13012-020-01070-3
117. Atkins L, Francis J, Islam R, O'Connor D, Patey A, Ivers N, et al. A guide to using the Theoretical Domains Framework of behaviour change to investigate implementation problems. Implement Sci. (2017) 12:77. doi: 10.1186/s13012-017-0605-9
118. Moffatt S, White M, Mackintosh J, Howel D. Using quantitative and qualitative data in health services research - what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Serv Res. (2006) 6:28. doi: 10.1186/1472-6963-6-28
Keywords: menu labelling, implementation science, implementation fidelity, Consolidated Framework for Implementation Research, mixed methods, multiple case study, triangulation
Citation: Kerins C, Kelly C, Reardon CM, Houghton C, Toomey E, Hayes CB, Geaney F, Perry IJ, McSharry J and McHugh S (2021) Factors Influencing Fidelity to a Calorie Posting Policy in Public Hospitals: A Mixed Methods Study. Front. Public Health 9:707668. doi: 10.3389/fpubh.2021.707668
Received: 10 May 2021; Accepted: 14 July 2021;
Published: 13 August 2021.
Edited by:
Ghada Soliman, City University of New York, United StatesReviewed by:
Rebecca Lee Hagedorn, Meredith College, United StatesJagmeet Madan, SNDT Women's University, India
Copyright © 2021 Kerins, Kelly, Reardon, Houghton, Toomey, Hayes, Geaney, Perry, McSharry and McHugh. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Claire Kerins, Yy5rZXJpbnMyJiN4MDAwNDA7bnVpZ2Fsd2F5Lmll; Y2xhaXJla2VyaW5zJiN4MDAwNDA7b3V0bG9vay5jb20=
†These authors have contributed equally to this work