Assessing sustainment of health worker outcomes beyond program end: Evaluation results from an infant and young child feeding intervention in Bangladesh

Introduction: Alive and Thrive (A&T) implemented infant and young child feeding (IYCF) interventions in Bangladesh. We examine the sustained impacts on health workers' IYCF knowledge, service delivery, job satisfaction, and job readiness three years after the program's conclusion.

Methods: We use data from a cluster-randomized controlled trial design, including repeated cross-sectional surveys with health workers in 2010 (baseline, n = 290), 2014 (endline, n = 511) and 2017 (post-endline, n = 600). Health workers in 10 sub-districts were trained and incentivized to deliver intensified IYCF counseling, and participated in social mobilization activities, while health workers in 10 comparison sub-districts delivered standard counseling activities. Accompanying mass media and policy change activities occurred at the national level. The primary outcome is quality of IYCF service delivery (number of IYCF messages reportedly communicated during counseling); intermediate outcomes are IYCF knowledge, job satisfaction, and job readiness. We also assess the role of hypothesized modifiers of program sustainment, i.e. activities of the program: comprehensiveness of refresher trainings and receipt of financial incentives. Multivariable difference-in-difference linear regression models, including worker characteristic covariates and adjusted for clustering at the survey sampling level, are used to compare differences between groups (intervention vs. comparison areas) and over time (baseline, endline, post-endline).

Results: At endline, health workers in intervention areas discussed significantly more IYCF topics than those in comparison areas (4.9 vs. 4.0 topics, p < 0.001), but levels decreased and the post-endline gap was no longer significant (4.0 vs. 3.3 topics, p = 0.067). Comprehensive refresher trainings were protective against deterioration in service delivery. Between baseline and endline, the intervention increased health workers' knowledge (3.5-point increase in knowledge scores in intervention areas, vs. 1.5-point increase in comparison areas, p < 0.0001); and this improvement persisted to post-endline, suggesting a sustained program effect on knowledge. Job satisfaction and readiness both saw improvements among workers in intervention areas during the project period (baseline to endline) but regressed to a similar level as comparison areas by post-endline.

Discussion: Our study showed sustained impact of IYCF interventions on health workers' knowledge, but not job satisfaction or job readiness—and, critically, no sustained program effect on service delivery. Programs of limited duration may seek to assess the status of and invest in protective factors identified in this study (e.g., refresher trainings) to encourage sustained impact of improved service delivery. Studies should also prioritize collecting post-endline data to empirically test and refine concepts of sustainment.

1. Introduction

Optimal breastfeeding and complementary feeding (defined as timely and adequate introduction of appropriate foods to an infant's diet in addition to breastmilk) strongly influence nutrition, growth and health outcomes in children (1, 2). Poor early-life nutrition contributes to malnutrition-related conditions (e.g., stunting) (3, 4) and delayed child development (5), and may cause up to an estimated 45% of all child deaths worldwide (6). Although some interventions to improve infant and young child feeding (IYCF)—i.e., optimal breastfeeding and complementary feeding—have achieved improvements, they have struggled during scale-up and sustainment (7, 8).

Sustaining components of effective interventions and program activities is essential for maintaining and supporting improvements in IYCF practices. Whether and how interventions are integrated into ongoing practices and institutions, for example through building capacity, can support sustainable implementation (6). However, there are many challenges to sustaining momentum or effective results with unpredictable funding resources that are largely donor driven, like limited multisectoral coordination and inadequate personal capacity (including high employee turnover) (9).

Sustainment can be conceptualized as ongoing activities that continue to result in improved outcomes (10). Sustainability of donor-funded programs is a crucial but understudied issue. As international organizations and donors increasingly wish to transition implementation responsibilities to recipient countries, and in some cases, ultimately phase out funding (11–13), it is imperative to better understand factors that enable (or hinder) lasting impacts from programs (10, 14–22). Data are lacking on the dynamics of outcomes affected by donor-funded programs after funding has ceased, and on factors that contribute to program sustainability (23, 24). Therefore, to advance the field of implementation science, research is needed evaluating longer-term outcomes of sustainability (e.g., sustainment) (25).

Alive & Thrive (A&T) supported nutrition interventions to improve maternal nutrition and infant and young child feeding practices in several countries including Burkina Faso, Ethiopia, India, Nigeria and Vietnam. In Bangladesh, A&T was implemented from 2009 to 2014 as a demonstration project of an at-scale model for achieving IYCF improvements (see Program Description, below). Findings from impact evaluation showed that A&T was associated with improved IYCF knowledge and behaviors: health workers in intervention areas had significantly greater improvements in IYCF knowledge and job motivation during the program period (2010 to 2014) relative to workers in comparison areas (26). A post-endline evaluation (conducted in 2017) of health workers in these same areas similarly found significantly better IYCF knowledge and job satisfaction in intervention vs. comparison areas (14, 27). While there is evidence that certain outcomes persisted beyond the end of the project period, data from baseline, endline, and post-endline have not been linked to estimate the degree of sustainment. Each evaluation effort deliberately aligned study samples and survey instruments, so it is possible to analyze the degree to which any intervention effects observed from 2009 to 2014 persisted until 2017.

In this paper, we investigate the presence and magnitude of “voltage drop”—i.e., attenuation of benefits over time (28)—after the A&T program in Bangladesh ended in 2014, i.e., the extent to which improvements in health worker outcomes in intervention areas were sustained or returned to the same level as in the comparison areas. This is an important area of study since capacity and resource limitations were identified as potential issues that could curtail scale-up and long-term improvements for A&T (29). Is there evidence of sustained differences in outcomes (quality of IYCF service delivery, IYCF knowledge, job satisfaction and job readiness) among health workers in intervention areas, vs. those in comparison areas, after the A&T initiative ended in Bangladesh? We also examine whether changes in these outcomes post-endline were differentially affected by program activities i.e., refresher training and receipt of financial incentives.

2. Materials and methods

2.1. Program description

Alive & Thrive (A&T) is an initiative supported by the Bill and Melinda Gates Foundation that aimed to demonstrate at-scale improvements in IYCF behaviors in Bangladesh, Ethiopia and Vietnam during Phase 1 from 2009 to 2014 [see detailed descriptions (26, 29, 30)]. Specific intervention details varied in each country but had a common core that included: interpersonal counseling, mass media, community mobilization, and policy advocacy activities– all bolstered by partnerships and strategic use of data (31, 32). In Bangladesh specifically, the A&T initiative included interpersonal communication (IYCF counseling with pregnant women and mothers of young children) and community mobilization (local meetings with stakeholders and village theater performances); these components were delivered by BRAC, a large non-governmental organization with a network of community-based volunteers, in areas assigned to A&T interventions. Mass media (television commercials and radio stories about IYCF) and policy advocacy (dissemination and the creation of a National IYCF Alliance) were also conducted country-wide across intervention and comparison A&T areas.

Here, we focus specifically on the interpersonal communication (IYCF counseling) and community mobilization components of the A&T program in Bangladesh. The unit of analysis are frontline health workers who deliver at-home health services to pregnant women and mothers of young children. We conceptualize the main outcome of interest as the quality of IYCF service delivery, operationalized as the quantity of IYCF topics discussed in counseling sessions. Quality of service delivery is imperative as it is associated with downstream improved IYCF outcomes among clients (33). We hypothesize that this is influenced by intermediate outcomes of IYCF knowledge, job readiness and job satisfaction—and that, in turn, these can be impacted by program activities of training and incentives (Figure 1).

FIGURE 1

Figure 1. Conceptual framework of the hypothesized relationship between Alive & Thrive Phase 1 program areas and activities, and outcomes among health workers (intermediate and main outcomes).

2.2. Study design

A&T Phase 1 was implemented using a cluster-randomized control trial design, and impact evaluation data were collected via cross-sectional surveys conducted in 2010 and 2014. There was a random selection of 20 rural upazillas (sub-districts) for the trial: 10 received the intervention A&T Phase 1 package of activities (the intervention areas) and 10 continued to implement standard care by BRAC frontline workers (comparison areas). These 20 upazillas were selected from among 357 upazillas in the participating 5 (of 7) divisions of Bangladesh. More information about the selection and randomization process has been previously published (14, 26, 34).

2.3. Data collection

Across the 20 participating upazillas in Bangladesh, 200 villages were randomly selected for data collection (from 3581 villages total); these 200 villages comprised the sampling frame for all three rounds of data collection. All health workers in these villages were surveyed in 2010 (baseline) and 2014 (endline). In 2017 (post-endline), these upazillas were revisited and health workers were randomly sampled using BRAC's rosters (ranging from 2 to 9, depending on the number available). To be eligible for participation in 2017, the health worker must have had a planned household visit within 2 days of initial contact by the study team, and must serve pregnant women and/or women with young children. A sample of 30 health workers per upazilla were invited to participate in the survey using random exclusion of eligible workers in the roster. A national total of 600 surveys (300 in intervention areas, 300 in comparison areas) were completed in 2017.

2.4. Variable definition

A summary of all variables included in the analysis is presented in Table 1.

TABLE 1

Table 1. Summary of variables included in analysis.

The primary outcome for this analysis was quality of IYCF service delivery, defined as the count of topics covered by the health worker during IYCF counseling (self-reported; range 0–14); this measure is available at endline and post-endline. Eligible IYCF topics that health workers could mention are listed in Supplementary Appendix Table S1.

We also assessed three intermediate outcomes hypothesized to be on the causal pathway between the A&T intervention and service delivery (Figure 1). First, we defined IYCF knowledge (both breastfeeding and complementary feeding) as the count of items answered correctly during the baseline, endline, and post-endline surveys (range 0–14). Items included were common across all three surveys (Supplementary Appendix Table S1). Second, we defined job satisfaction as a binary variable (very satisfied vs. not very satisfied). At baseline and endline, this was measured using a 10-point scale and “very satisfied” was operationalized as scores ≥8 on this scale; at post-endline, this was measured using a 5-point scale, and “very satisfied” was defined as reporting ≥4. Third, job readiness was a binary variable representing whether the health worker felt they had adequate training for their job (binary yes/no variable).

We hypothesized that two A&T intervention activities could modify sustainment of effects: (1) comprehensiveness of refresher training, defined as the proportion of self-reported topics a health worker recalled receiving at their last refresher training (measured only at endline and post-endline); and (2) receipt of incentives, measured through two variables: a binary variable representing whether the health worker received incentives in the last year, and a continuous variable representing the amount of incentive received in the last year (with 0 if no incentive was received). Both financial incentive variables were measured only at endline and post-endline.

We adjusted for a number of covariates that could impact IYCF service delivery and its sustainment over time, including heath worker cadre (binary: standard health workers defined as Shasthya Shebika [SS]/Pushti Shebika [PS], vs. higher-level health workers including Shasthya Kormi [SK]/Pushti Kormi [PK]), age in years (continuous), years in job (continuous), and education level (categorical: none, some or completed primary, some or completed secondary, and higher than secondary).

2.5. Statistical analysis

We specified multivariable linear regression models (for continuous outcomes, including quality of service delivery, knowledge score, and incentive amount), and linear probability models (for binary outcomes, including job satisfaction, job readiness, and whether the health worker received an incentive in the previous 12 months). Linear probability models were utilized for binary outcomes to aid in ease of interpretation of findings. Difference-in-difference (DID) estimates were generated by interacting an indicator variable representing intervention areas (vs. comparison areas) and an indicator variable representing time (baseline, endline, and post-endline, with endline as the reference time point). To assess effect modification by refresher training quality and financial incentives, we included three-way interaction terms (intervention vs. comparison, and the hypothesized modifier), with lower order (two-way) interactions and main effects included. To visualize the three-way interaction terms, we dichotomized the refresher training variable using the 10th and 90th percentile of topics covered during training. For each model, we generated marginal predicted probabilities of the outcome at each time point in intervention and comparison areas. These marginal predicted probabilities allowed us to compare between-group differences (intervention vs. control) at each time point. Furthermore, we assessed whether the slope differed between each time point for intervention and control. Within-group changes in the post-endline period should be interpreted as exploratory. All models included robust standard errors allowing for intragroup correlation clustered at the upazilla (intervention) level. Models were adjusted for health worker cadre, age, years in job, and education level. Analyses were carried out in Stata v17.

3. Results

A total of n = 290 health workers were surveyed at baseline (147 in intervention areas, 143 in comparison areas), n = 511 at endline (347 in intervention areas, 164 in comparison areas), and n = 600 at post-endline (300 in each area) (Table 2). Most respondents were SS/PS cadre and had an average number of 5–10 years in their current role. The sample was not fully balanced at endline and post-endline: respondents in comparison areas had more years of experience, and educational attainment was different; respondents from the comparison were also slightly older at endline and there was a difference in the percentage of cadre represented at post-endline (Supplementary Appendix Table S2).

TABLE 2

Table 2. Sample characteristics among frontline health workers at each survey round.

3.1. Sustainment of program activities and outcomes at each round

3.1.1. Primary outcome

The program effect was not sustained for quality of IYCF service delivery, i.e., number of IYCF topics discussed during care. As shown in Table 3 (comparing the outcome at each time point for intervention vs. control) and Figure 2 (demonstrating the differences in the changes in the outcome over time), the adjusted marginal predictions of self-reported number of IYCF topics discussed during IYCF counseling visits was significantly higher among health workers in intervention areas than comparison areas at endline (4.93 [SE 0.10] topics covered by health workers in intervention areas and 3.97 [SE 0.18] topics covered by health workers in comparison areas). By post-endline, delivery of IYCF messages declined among both intervention and comparison health workers, with no significant difference in this decline (DID p = 0.32). The difference between intervention and comparison areas was marginally significant at post-endline (3.97 [SE 0.18] topics covered by health workers in intervention areas and 3.34 [SE 0.28] topics covered by health workers in comparison areas).

FIGURE 2

Figure 2. Changes in delivery of the quantity of IYCF messages included during counseling by area over time (difference in difference estimates).

TABLE 3

Table 3. Primary and intermediate outcomes and program activities at each survey round: marginal predicted outcomes resulting from difference-in-difference models (SE in parentheses); p-values for the null hypothesis of no difference in predicted outcomes in intervention and comparison areas at each round.

3.1.2. Intermediate outcomes

Findings for intermediate outcomes over time are presented in Table 3 (comparing the outcome at each time point for intervention vs. comparison) and Figure 3 (demonstrating differences in changes in the intermediates outcomes over time). At baseline, there were no substantive differences between all intermediate outcomes—IYCF knowledge, job satisfaction, and job readiness—comparing intervention and comparison areas. At endline, significantly better IYCF knowledge was observed among all health workers in both intervention and comparison areas compared to baseline, but gains were significantly larger among health workers in intervention areas. There was evidence of a sustained program effect for IYCF knowledge as these improvements in intervention areas persisted to post-endline. The gap between intervention and comparison areas remained largely unchanged by post-endline (IYCF knowledge score of 12.65 [SE 0.95] among intervention health workers vs. 11.13 [SE 0.050] among comparison workers), with no detectable difference in the slope for these changes between endline and post-endline.

FIGURE 3

Figure 3. Changes in intermediate outcomes (IYCF knowledge, job satisfaction, job readiness) by area over time (difference in difference estimates).

In comparison areas, job satisfaction declined across all periods. In intervention areas, job satisfaction increased during the project period (baseline to endline); however, there was no detectable difference in the changes over time (DID p = 0.29). Subsequently, job satisfaction among workers in intervention areas declined at post-endline and reached a similar level as comparison areas (48% [SE 4.4] satisfied health workers in intervention areas and 38% [SE 3.8] satisfied health workers in comparison areas).

In intervention areas, health workers experienced an increase in reported job readiness by endline, while there was no change among workers in comparison areas (DID p = 0.014). However, by post-endline intervention-area workers had returned to their baseline level, while health workers in comparison areas reported no change (DID p = 0.002).

3.1.3. Program activities

Sustainment of program activities are presented in Table 3 (comparing the outcome at each time point for intervention vs. comparison) and Figure 4 (demonstrating differences in changes in the program activities over time). Despite declining from endline to post-endline (DID p = 0.002), IYCF topics covered at last refresher training remained significantly higher in intervention areas at post-endline vs. comparison areas. Incentive payments declined substantially from endline to post-endline among both intervention and comparison health workers, with a steeper decline among those in intervention areas (DID p < 0.001). At endline, incentive payments were significantly more common in intervention areas: 90% of workers in intervention areas and 46% of workers in comparison areas said they had received an incentive payment. At post-endline, these values had fallen to 5% and 11%, respectively.

FIGURE 4

Figure 4. Changes in program activities (refresher trainings, incentives received, incentive amount) by area over time (difference in difference estimates).

3.2. Factors modifying the sustainment of program effects

We assessed whether program activities modified the sustainment of program effects of the primary outcome (IYCF service delivery, Figure 5).

FIGURE 5

Figure 5. Changes in delivery of IYCF messages during counseling by area over time, including interaction terms for program activities.

Comprehensive refresher trainings potentially protected against the deterioration of the primary outcome (service delivery): workers who reported more comprehensive refresher trainings (above the 90th percentile of reported topics discussed) delivered significantly more IYCF messages during counseling by post-endline, in both intervention and comparison areas. Workers in intervention areas who had less-comprehensive refresher trainings experienced the largest declines in IYCF counseling by post-endline. However, more comprehensive IYCF refresher training was not associated with any difference in knowledge (intermediate outcome) across treatment groups, between endline and post-endline (Supplementary Appendix Figure S1). There was however no apparent interaction between comprehensiveness of refresher training, and job satisfaction or job readiness, in either group over time (Supplementary Appendix Figures S2, S3).

There was no apparent effect of incentives, whether presence of any incentive or the amount of incentive, on quality of IYCF service delivery nor on intermediate outcomes (Figure 5 and Supplementary Appendix Figures S1–S3), with the exception of job readiness—health workers in intervention areas with no incentive reported lower job readiness at endline, but converged with their peers at post-endline.

4. Discussion

This paper adds to the empirical literature on sustainability as there are relatively few quantitative studies that estimate the degree to which program outcomes endure over time (19). We find mixed evidence about the sustainment of outcomes from the Alive & Thrive initiative three years after the program's conclusion in Bangladesh. On the main outcome of IYCF service delivery and quality (i.e., number of topics discussed during counseling), we see evidence of “voltage drop” (28): the intervention was associated with a significant improvement in service delivery at endline, but three years later, quality had declined for workers in both intervention and comparison areas and was no longer significantly different between the groups.

Importantly, we find that although there were not enduring improvements in intervention areas for the main outcome of service quality, a critical intermediate outcome—IYCF knowledge—was sustained, as was an associated related program activity of IYCF topics covered at refresher trainings. This indicates that interventions that seek to improve health worker knowledge may have an enduring effect but that this may not continue to impact behaviors. This adds a new dimension to the “know-do gap” literature, which has demonstrated the potential disconnect between what health workers know and their clinical practice (35–39). Prior IYCF research in Ethiopia found mothers with better access to nutrition education had higher knowledge scores, improved child feeding practices, and reduced rates of stunting among their children (40). However, while knowledge may be a necessary ingredient to achieving—and sustaining—behavior change, it may not be sufficient. Future research should assess health workers' knowledge-sharing efficacy and other factors that may also impact IYCF service delivery and quality (41).

Two key intermediate outcomes—job satisfaction and job readiness—increased over the program period but this improvement was not sustained. Our previous research has indicated that this may be directly attributable to the end of A&T: removal of incentives has been shown to negatively impact BRAC health workers' IYCF service delivery quality (42) and desire to perform (43).

Alive & Thrive was designed as a “proof of concept” initiative: if it demonstrated success, then governments would have the evidence necessary to implement it. Although some activities have endured in Bangladesh, not all have continued in the format or intensity as in the initial design (14). Externally-funded programs and projects cannot continue forever; these findings add new insights to the growing literature on sustainability by measuring sustainment across a program's theory of change. If, as in this case, an intervention can have lasting effects on health worker knowledge, but cannot continue to make material contributions that may help translate this knowledge into action—for example, incentive payments or job readiness—what are reasonable sustainability expectations? It is important to consider this finding in the context of the study population: BRAC community health workers include both paid and unpaid (volunteers), and different types of health workers may differentially respond to programs, and to their end (44, 45). Further, there may be other factors—including macroeconomic factors or structural changes in the healthcare system—that may impact intermediate outcomes, like job satisfaction for health workers. Previous research has explored how social factors, like religious norms, may affect BRAC workers' job performance (46). It is important to consider contextual factors like these when preparing for and assessing sustainability.

This is also particularly noteworthy when considering our exploratory finding that refresher trainings may protect against the “voltage drop” in service quality. More research that attempts to disentangle sustainment of effects across a program's theory of change may help policymakers prioritize areas for continued investment after programs end, in order to catalyze ongoing impacts.

This analysis is innovative in its design and approach, but is not without limitations. First, the primary outcome—number of IYCF topics discussed during recent counseling visits—is not a perfect measure of service quality. It was self-reported and blinding of the intervention was impossible so health workers in intervention areas so may have over-reported their performance. Additionally, IYCF counseling is not a “one-size-fits-all” activity and the topics discussed will naturally differ across clients, so more topics is not necessarily a measure of high-quality counseling. There were also measurement challenges with this variable owing to different recall periods: no recall period was specified at endline, but at post-endline health workers were only asked about counseling in the last 30 days. This may account for at least some of the decline in messages reportedly delivered. Second, some outcomes were measured only at endline and post-endline, so we cannot assess whether the observed declines represent returns to baseline levels. Third, response options varied slightly between survey rounds; for example, the job satisfaction variable used 10-point Likert scale in the baseline and endline surveys, but a 5-point scale in the post-endline survey. This may account for some difference in responses over time. Fourth, this analysis was not powered to detect changes over time, nor to assess effect modification over time. Nevertheless, the findings do indicate evidence of “voltage drop,” and suggest some factors that may protect against declines in program sustainment. To assess whether the hypothesized effect modifiers could instead be mediators, we conducted a mediation analysis using Barron and Kenney methods to test whether the difference-in-difference coefficient was statistically different with and without potential mediators in the model—however the results showed no statistical difference. Fifth, this analysis focused solely on outcomes among health workers and did not include key IYCF measures among caregivers of children, such as exclusive breastfeeding; although this is a limitation, our previous research has found that service quality (IYCF messages delivered during counseling) is associated with improved IYCF outcomes in this population (33). Lastly, some outcomes may have been impacted by social desirability bias (such as job satisfaction and job readiness); however, we do not expect this bias to differ between intervention and comparison areas nor over time.

This was a unique analysis: we linked three separate cross-sectional surveys—performed over a 7-year period and spanning intervention baseline, endline and post-endline—to quantitatively examine sustainment of program outcomes and activities. The study teams deliberately aligned survey tools and sampling frames in order to enable this analysis, and both the study design and its limitations offer useful lessons for scholars of sustainability. We find evidence of “voltage drop” in the primary outcome of quality of service delivery, although refresher trainings may protect against this deterioration. There are sustained improvements in knowledge among health workers in intervention areas compared to their counterparts in comparison areas, but this was not sufficient to achieve sustained outcomes. We hope this analysis stimulates more research to empirically measure and quantify sustainment, particularly across interventions' theories of change and into post-endline periods.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: https://dataverse.harvard.edu/dataverse/AliveandThrive.

Ethics statement

The studies involving human participants were reviewed and approved by the Bangladesh Medical Research Council and the Institutional Review Board at the International Food Policy Research Institute (baseline and endline); and the institutional Review Boards at the Harvard T.H. Chan School of Public Health, the University of California Los Angeles, and the icddr,b in Bangladesh (post-endline). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

The post-endline study was led by CM and TJB, with data collection led by HS, MR and MT, and data analysis and research assistance from AE, JG and DDP. The baseline and endline studies were led by SSK, PHN and PM. This combined analysis was led by CM and AE with design inputs from SSK, PHN, JG, DDP, PM and TJB. All authors reviewed the results and contributed to the interpretation. CM and AE wrote the first draft of the manuscript, which all authors reviewed and gave feedback. All authors contributed to the article and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frhs.2022.1005986/full#supplementary-material.

References

1. Bhutta ZA, Das JK, Rizvi A, Gaffey MF, Walker N, Horton S, et al. Evidence-based interventions for improvement of maternal and child nutrition: what can be done and at what cost? Lancet. (2013) 382(9890):452–77. doi: 10.1016/S0140-6736(13)60996-4

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Victora CG, Bahl R, Barros AJ, França GV, Horton S, Krasevec J, et al. Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect. Lancet. (2016) 387(10017):475–90. doi: 10.1016/S0140-6736(15)01024-7

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Black RE, Victora CG, Walker SP, Bhutta ZA, Christian P, De Onis M, et al. Maternal and child undernutrition and overweight in low-income and middle-income countries. Lancet. (2013) 382(9890):427–51. doi: 10.1016/S0140-6736(13)60937-X

PubMed Abstract | CrossRef Full Text | Google Scholar

4. De Onis M, Branca F. Childhood stunting: a global perspective. Matern Child Nutr. (2016) 12:12–26. doi: 10.1111/mcn.12231

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Grantham-McGregor S, Cheung YB, Cueto S, Glewwe P, Richter L, Strupp B, et al. Developmental potential in the first 5 years for children in developing countries. Lancet. (2007) 369(9555):60–70. doi: 10.1016/S0140-6736(07)60032-4

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Black RE, Alderman H, Bhutta ZA, Gillespie S, Haddad L, Horton S, et al. Maternal and child nutrition: building momentum for impact. Lancet. (2013) 382(9890):372–5. doi: 10.1016/S0140-6736(13)60988-5

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Bhutta ZA, Labbok M. Scaling up breastfeeding in developing countries. Lancet. (2011) 378(9789):378–80. doi: 10.1016/S0140-6736(11)60897-0

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Gillespie S, Menon P, Kennedy AL. Scaling up impact on nutrition: what will it take? Adv Nutr. (2015) 6(4):440–51. doi: 10.3945/an.115.008276

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Gillespie S, Haddad L, Mannar V, Menon P, Nisbett N, Maternal and Child Nutrition Study Group. The politics of reducing malnutrition: building commitment and accelerating progress. Lancet. (2013) 382(9891):552–69. doi: 10.1016/S0140-6736(13)60842-9

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Scheirer MA, Dearing JW. An agenda for research on the sustainability of public health programs. Am J Public Health. (2011) 101(11):2059–67. doi: 10.2105/AJPH.2011.300193

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Moucheraud C, Sparkes S, Nakamura Y, Gage A, Atun R, Bossert TJ. Pepfar investments in governance and health systems were one-fifth of Countries’ budgeted funds, 2004–14. Health Aff. (2016) 35(5):847–55. doi: 10.1377/hlthaff.2015.1445

CrossRef Full Text | Google Scholar

12. Vogus A, Graff K. Pepfar transitions to country ownership: review of past donor transitions and application of lessons learned to the eastern Caribbean. Glob Health Sci Pract. (2015) 3(2):274–86. doi: 10.9745/GHSP-D-14-00227

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Resch S, Hecht R. Transitioning financial responsibility for health programs from external donors to developing countries: key issues and recommendations for policy and research. J Glob Health. (2018) 8(1):010301. doi: 10.7189/jogh.08.010301

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Moucheraud C, Sarma H, Ha TTT, Ahmed T, Epstein A, Glenn J, et al. Can complex programs be sustained? A mixed methods sustainability evaluation of a national infant and young child feeding program in Bangladesh and Vietnam. BMC Public Health. (2020) 20(1):1–14. doi: 10.1186/s12889-020-09438-2

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Biesma RG, Brugha R, Harmer A, Walsh A, Spicer N, Walt G. The effects of global health initiatives on country health systems: a review of the evidence from hiv/aids control. Health Policy Plan. (2009) 24(4):239–52. doi: 10.1093/heapol/czp025

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Bossert TJ. Can they get along without us? Sustainability of donor-supported health projects in Central America and Africa. Soc Sci Med. (1990) 30(9):1015–23. doi: 10.1016/0277-9536(90)90148-L

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Bowser D, Sparkes SP, Mitchell A, Bossert TJ, Bärnighausen T, Gedik G, et al. Global fund investments in human resources for health: innovation and missed opportunities for health systems strengthening. Health Policy Plan. (2014) 29(8):986–97. doi: 10.1093/heapol/czt080

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Gruen RL, Elliott JH, Nolan ML, Lawton PD, Parkhill A, McLaren CJ, et al. Sustainability science: an integrated approach for health-programme planning. Lancet. (2008) 372(9649):1579–89. doi: 10.1016/S0140-6736(08)61659-1

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Shelton RC, Cooper BR, Stirman SW. The sustainability of evidence-based interventions and practices in public health and health care. Annu Rev Public Health. (2018) 39:55–76. doi: 10.1146/annurev-publhealth-040617-014731

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. Am J Eval. (2005) 26(3):320–47. doi: 10.1177/1098214005278752

CrossRef Full Text | Google Scholar

21. Stirman SW, Kimberly J, Cook N, Calloway A, Castro F, Charns M. The sustainability of new programs and innovations: a review of the empirical literature and recommendations for future research. Implement Sci. (2012) 7(17):1–19. doi: 10.1186/1748-5908-7-17

CrossRef Full Text | Google Scholar

22. Escobar-Alegria JL, Frongillo EA, Blake CE. How country policy actors understand sustainability of food and nutrition security policy. Glob Food Sec. (2022) 32:100603. doi: 10.1016/j.gfs.2021.100603

CrossRef Full Text | Google Scholar

23. Hailemariam M, Bustos T, Montgomery B, Barajas R, Evans LB, Drahota A. Evidence-based intervention sustainability strategies: a systematic review. Implement Sci. (2019) 14(1):57. doi: 10.1186/s13012-019-0910-6

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Iwelunmor J, Blackstone S, Veira D, Nwaozuru U, Airhihenbuwa C, Munodawafa D, et al. Toward the sustainability of health interventions implemented in sub-saharan Africa: a systematic review and conceptual framework. Implement Sci. (2015) 11(1):43. doi: 10.1186/s13012-016-0392-8

CrossRef Full Text | Google Scholar

25. Chambers DA. Commentary: increasing the connectivity between implementation science and public health: advancing methodology, evidence integration, and sustainability. Annu Rev Public Health. (2018) 39:1–4. doi: 10.1146/annurev-publhealth-110717-045850

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Menon P, Nguyen PH, Saha KK, Khaled A, Kennedy A, Tran LM, et al. Impacts on breastfeeding practices of at-scale strategies that combine intensive interpersonal counseling, mass Media, and community mobilization: results of cluster-randomized program evaluations in Bangladesh and viet nam. PLoS Med. (2016) 13(10):e1002159. doi: 10.1371/journal.pmed.1002159

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Kim SS, Nguyen PH, Tran LM, Sanghvi T, Mahmud Z, Haque MR, et al. Large-scale social and behavior change communication interventions have sustained impacts on infant and young child feeding knowledge and practices: results of a 2-year follow-up study in Bangladesh. J Nutr. (2018) 148(10):1605–14. doi: 10.1093/jn/nxy147

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. (2013) 8(1):117. doi: 10.1186/1748-5908-8-117

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Sanghvi T, Haque R, Roy S, Afsana K, Seidel R, Islam S, et al. Achieving behaviour change at scale: alive & thrive’s infant and young child feeding programme in Bangladesh. Matern Child Nutr. (2016) 12:141–54. doi: 10.1111/mcn.12277

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Nguyen P, Kim S, Tran L, Menon P, Frongillo E. Intervention design elements are associated with frontline health Workers’ performance to deliver infant and young child nutrition services in Bangladesh and viet nam (P10-128-19). Curr Dev Nutr. (2019) 3(Suppl. 1):nzz034, P10-128-19. doi: 10.1093/cdn/nzz070

CrossRef Full Text | Google Scholar

31. Nguyen PH, Menon P, Keithly SC, Kim SS, Hajeebhoy N, Tran LM, et al. Program impact pathway analysis of a social franchise model shows potential to improve infant and young child feeding practices in Vietnam. J Nutr. (2014) 144(10):1627–36. doi: 10.3945/jn.114.194464

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Rawat R, Nguyen PH, Ali D, Saha K, Alayon S, Kim SS, et al. Learning how programs achieve their impact: embedding theory-driven process evaluation and other program learning mechanisms in alive & thrive. Food Nutr Bull. (2013) 34(3_suppl2):S212–S25. doi: 10.1177/15648265130343S207

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Epstein A, Moucheraud C, Sarma H, Rahman M, Tariqujjaman M, Ahmed T, et al. Does health worker performance affect Clients’ health behaviors? A multilevel analysis from Bangladesh. BMC Health Serv Res. (2019) 19(1):1–9. doi: 10.1186/s12913-019-4205-z

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Menon P, Rawat R, Ruel M. Bringing rigor to evaluations of large-scale programs to improve infant and young child feeding and nutrition: the evaluation designs for the alive & thrive initiative. Food Nutr Bull. (2013) 34(3_suppl2):S195–S211. doi: 10.1177/15648265130343S206

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Pakenham-Walsh N. Learning from one another to bridge the” know-do gap”. BMJ. (2004) 329(7475):1189. doi: 10.1136/bmj.329.7475.1189

CrossRef Full Text | Google Scholar

36. Haines A, Kuruvilla S, Borchert M. Bridging the implementation gap between knowledge and action for health. Bull World Health Organ. (2004) 82(10):724–31.15643791

PubMed Abstract | Google Scholar

37. Das J. The quality of medical care in low-income countries: from providers to markets. PLoS Med. (2011) 8(4):e1000432. doi: 10.1371/journal.pmed.1000432

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Mohanan M, Vera-Hernández M, Das V, Giardili S, Goldhaber-Fiebert JD, Rabin TL, et al. The know-do gap in quality of health care for childhood diarrhea and pneumonia in rural India. JAMA Pediatr. (2015) 169(4):349–57. doi: 10.1001/jamapediatrics.2014.3445

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Gage AD, Kruk ME, Girma T, Lemango ET. The know-do gap in sick child care in Ethiopia. PLoS ONE. (2018) 13(12):e0208898. doi: 10.1371/journal.pone.0208898

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Abebe Z, Haki GD, Baye K. Health extension workers’ knowledge and knowledge-sharing effectiveness of optimal infant and young child feeding are associated with mothers’ knowledge and child stunting in rural Ethiopia. Food Nutr Bull. (2016) 37(3):353–63. doi: 10.1177/0379572116651209

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Mbuya MN, Menon P, Habicht J-P, Pelto GH, Ruel MT. Maternal knowledge after nutrition behavior change communication is conditional on both health workers’ knowledge and knowledge-sharing efficacy in rural Haiti. J Nutr. (2013) 143(12):2022–8. doi: 10.3945/jn.113.178442

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Tariqujjaman M, Rahman M, Luies SK, Karmakar G, Ahmed T, Sarma H. Unintended consequences of programmatic changes to infant and young child feeding practices in Bangladesh. Matern Child Nutr. (2021) 17(2):e13077. doi: 10.1111/mcn.13077

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Glenn J, Moucheraud C, Payán DD, Crook A, Stagg J, Sarma H, et al. What is the impact of removing performance-based financial incentives on community health worker motivation? A qualitative study from an infant and young child feeding program in Bangladesh. BMC Health Serv Res. (2021) 21(1):1–11. doi: 10.1186/s12913-021-06996-y

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Alam K, Tasneem S, Oliveras E. Performance of female volunteer community health workers in Dhaka urban slums. Soc Sci Med. (2012) 75(3):511–5. doi: 10.1016/j.socscimed.2012.03.039

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Sarma H, Tariqujjaman M, Mbuya MN, Askari S, Banwell C, Bossert TJ, et al. Factors associated with home visits by volunteer community health workers to implement a home-fortification intervention in Bangladesh: a multilevel analysis. Public Health Nutr. (2021) 24(S1):s23–s36. doi: 10.1017/S1368980019003768

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Sarma H, Jabeen I, Luies SK, Uddin MF, Ahmed T, Bossert TJ, et al. Performance of volunteer community health workers in implementing home-fortification interventions in Bangladesh: a qualitative investigation. PLoS One. (2020) 15(4):e0230709. doi: 10.1371/journal.pone.0230709

PubMed Abstract | CrossRef Full Text | Google Scholar