A Combined Theory of Change-Group Model Building Approach to Evaluating “Farm to Fork” Models for School Feeding in the Caribbean

Saint Ville, Arlette; Hickey, Gordon M.; Rouwette, Etiënne; Samuels, Alafia; Guariguata, Leonor; Unwin, Nigel; Phillip, Leroy E.

doi:10.3389/fsufs.2022.801731

ORIGINAL RESEARCH article

Front. Sustain. Food Syst., 05 April 2022

Sec. Nutrition and Sustainable Diets

Volume 6 - 2022 | https://doi.org/10.3389/fsufs.2022.801731

A Combined Theory of Change-Group Model Building Approach to Evaluating “Farm to Fork” Models for School Feeding in the Caribbean

$\nArlette Saint Ville,$ Arlette Saint Ville^1,2

Gordon M. Hickey²^*

Etiënne Rouwette³

Alafia Samuels⁴

Leonor Guariguata⁴

Nigel Unwin^5,6

Leroy E. Phillip²

¹Faculty of Food and Agriculture, The University of the West Indies, St. Augustine, Trinidad and Tobago
²Faculty of Agricultural and Environmental Sciences, McGill University, Montreal, QC, Canada
³Institute for Management Research, Radboud University, Nijmegen, Netherlands
⁴George Alleyne Chronic Disease Research Centre, The University of the West Indies, Cave Hill Campus, Bridgetown, Barbados
⁵MRC Epidemiology Unit, University of Cambridge, Cambridge, United Kingdom
⁶European Centre for Environment and Human Health, University of Exeter, Truro, United Kingdom

There is a scarcity of research on building nutrition-sensitive value chains (NSVCs) to improve diets and nutrition outcomes of populations in the Caribbean. This study contributes to filling this research gap by outlining a participatory approach to evaluating a NSVC model for “farm to fork” (F2F) school feeding in the Eastern Caribbean Island of St. Kitts. Using a combined group model building (GMB) and theory of change (ToC) approach, policy actors and other stakeholders (n = 37) across the school feeding value chain were guided through a facilitated process to evaluate the ToC underlying a series of F2F interventions designed to enhance childhood nutrition. Stakeholders at the workshop engaged collaboratively to create a causal map of interconnected “system factors” that help explain behaviors contributing to unhealthy eating among children that extended well-beyond the original F2F project ToC that had been used to inform interventions. Through this facilitated GMB process, stakeholders proposed additional food system interventions, and identified multiple “impact pathways” and “mediating influences” underlying local availability and consumption of nutritious foods in local school environments. Workshop participants were also able to identify leverage points where community-level efforts, alongside research interventions, may ensure that initiatives for building local NSVCs are ultimately institutionalized. Results of this study suggest that developing NSVCs for school feeding and food systems in the Caribbean requires both locally driven innovation and the leveraging of system-wide resources, with lessons for project intervention strategies.

Introduction

Caribbean Small Island Developing States (SIDS) are experiencing high rates of overweight and obesity and associated high burdens of chronic non-communicable disease (NCDs) due, in part, to unhealthy dietary patterns (Hospedales et al., 2011; Sobers and Samuels, 2019; Rambaran et al., 2021; Thomas and Theokritoff, 2021). Childhood obesity in the region is of particular concern (Batson et al., 2014; Gaskin et al., 2015; Mumena et al., 2018) because obese children are more likely to become obese adults with an increased risk of suffering from NCDs later in life (Ng et al., 2014). A wide range of efforts have been identified as being needed to respond to the challenge of unhealthy diets in the Caribbean, with work required to understand ongoing food system changes, how to support local food production, devise context-specific interventions along complex food value chains to enhance nutrition, and how to evaluate outcomes (Saint Ville et al., 2015; Ford et al., 2017; Lowitt et al., 2018; Connell et al., 2020).

Food value chains include all actors and activities from food production to consumption (from “farm to fork”), and therefore offer a potentially useful approach to incorporating complexity into problem solving (de Brauw et al., 2015; Lowitt et al., 2015a). Nutrition sensitive value chains (NSVCs) have become an increasingly common research and policy objective internationally [Food and Agriculture Organization (FAO), 2014, 2015, 2017; de la Pena et al., 2018], involving the careful design of interventions through a range of actors along an identified value chain to ensure nutritious products reach consumers (de Brauw et al., 2015; Gelli et al., 2015; Allen and de Brauw, 2019). While there is a growing and substantial body of research on increasing and evaluating nutrition outputs in value chains (Ruel et al., 2013; Gelli et al., 2015, 2019; Donovan and Gelli, 2019; Nicholson et al., 2021), consumption-oriented research assessing demand-side constraints has so far been marginal (Donovan and Gelli, 2019; Morgan et al., 2019), particularly in the Caribbean.

School feeding programs (SFPs) have been identified as potentially useful entry points for developing NSVCs to support the availability of more diverse diets internationally (Allen and de Brauw, 2019). The operating principle is that school foods are sourced from local smallholder farmers [(World Food Programme (WFP), 2017)] and then the system is supported with behavior change communication and new institutions to improve nutrition, support local agriculture and improve children's attendance at schools (Allen and de Brauw, 2018). In a recent report on SFPs in 14 countries in the Caribbean Community (CARICOM), FAO and The University of the West Indies (FAO and University of the West Indies, 2021) emphasized the importance of linking local smallholder farmers to SFPs to provide children with diverse, nutritious, and culturally appropriate meals. The report concluded that the expansion of SFPs in the CARICOM would benefit children, local food producers and the regional economy. According to Hawkes et al. (2020), efforts to build NSVCs for school feeding should be oriented around the identified nutritional needs, demand and supply constraints in the food system under study, and the development of interventions to address them. However, this requires supportive institutional structures capable of enabling social learning through two-way knowledge exchange among value chain actors (Lowitt et al., 2015a). This is a major challenge in the Caribbean, where local producers and smallholder farming systems are not well-organized, and the supportive institutions, financial resources, and social infrastructure required to implement NSVSs are not necessarily available or well-coordinated (Lowitt et al., 2015b; Saint Ville et al., 2015, 2016, 2017). Further, the nutrition impact of NSVC interventions can be difficult to observe, undermining efforts to institutionalize complex value chain interventions (Lowitt et al., 2015a, 2018; de la Pena et al., 2018). de Brauw et al. (2015) have reported a lack of formal evaluation and supporting evidence as knowledge gaps that limit understanding of the performance, complexity, and feasibility of “farm to fork” NSVC approaches. To date, there has been almost no evaluative research designed to inform the multi-actor and multi-level strategies for institutionalizing desired interventions in the day-to-day practices of value chain actors, and what such insights could offer future intervention design and engagement of communities. According to Cosens et al. (2017), complex systems-based approaches are likely to be helpful when evaluating NSVCs, offering an opportunity to reflect on past interventions and enhance the capacity of actors to adapt the design of new processes based on shared understandings of how outcomes occur within the system (see also Guariguata et al., 2020; McGill et al., 2020; Nicholson et al., 2021). In this paper, we present the results of a participatory process evaluation of a large farm to fork school feeding project, undertaken in the Caribbean Island of St. Kitts, using methods informed by complex systems thinking. We sought to develop a shared understanding of how intervention-related outcomes occur within school feeding systems in the Caribbean and how local capacity can be leveraged to co-design interventions that facilitate institutional uptake of farm to fork school feeding objectives in public policy.

Study Setting and Background

The study was undertaken on the island of St. Kitts, one of the twin-island federation of St. Kitts and Nevis (SKN) located in the northeast of the Caribbean archipelago. The smallest sovereign state in the Americas, St. Kitts and Nevis has a population of ~55,000 and land base of 269 square km. The federation is a member of the CARICOM, an economic grouping of 15 member countries (mainly Small Island Developing States), all experiencing high prevalence of adult and childhood obesity, and non-communicable diseases [PAHO, 2011; Pan American Health Organization (PAHO), 2012; Sobers and Samuels, 2019; Rambaran et al., 2021]. Published studies reveal low levels of dietary diversity among the SKN population, high dependence on imported, energy-dense and ultra-processed foods [Food and Agriculture Organization (FAO), 2013], and increasing prevalence of overweight and obesity among children (Mumena et al., 2018; Rambaran et al., 2021).

Since the 1980's, the St. Kitts government funds and operates a centralized School Meals Centre (SMC) that provides a free hot lunch daily to ~5,000 children in 18 primary schools and 200 deserving students in six secondary schools (FAO and University of the West Indies, 2021). Lunch meals prepared by the SMC are delivered to schools in bulk-holding containers. Food service staff at the schools plate and serve meals to the children, who consume the meals in lunchrooms or classrooms (FAO and University of the West Indies, 2021). The meals are based primarily on imported food ingredients of low nutritional value (Lowitt et al., 2018) and the level of procurement and contribution of locally grown produce to the school meals menu is low. These local nutritional gaps and food system challenges provided the impetus for a large, multidisciplinary “farm to fork” (F2F) research project; its aim was to develop and implement a local NSVC to improve the nutritional quality of school meals by increasing the portions of locally grown vegetables and fruits offered to the children, while strengthening local agricultural production practices (Lowitt et al., 2018). The project was undertaken over a period of 18 months (from January 2013 to June 2014) and seven primary schools, and 16 smallholder crop farmers participated in the project (Lowitt et al., 2018).

The project's theory of change (ToC), designed by a team of international and regional researchers, posited that, getting school children to consume more fruits and vegetables, would require increased local production and supply of these fresh nutritious food items to the school meals program. The theory of change assumed that providing small-holder farmers with technology and training to more effectively produce nutrition-dense crops that were familiar to farmers and could be easily integrated into the Caribbean food culture, would lead to an increase in the supply and diversity of the basket of local fresh food items for preparing school meals. This in turn would improve the nutritional quality of school lunch menus and build local capacity for the delivery of safe and nutritious meals, ultimately leading to improvements in healthy eating among children.

The F2F ToC is visualized in Figure 1, depicting the areas of intervention and anticipated outcomes. The project attempted to pilot the building of a local NSVC that would function within the existing SFP in St. Kitts. The research was led collaboratively by McGill University and The University of the West Indies (UWI), in partnership with the local government and regional agricultural institutions. Local SFP staff and government officials collaborated and coordinated with project researchers, who also worked with farmers and extension officers to improve production and delivery of local produce (primarily fruits and vegetables) to the SMC. Working from existing lunch menus, professional nutritionists from the F2F project designed “test” menus that incorporated the locally grown produce into nutritionally balanced lunch meals for 5–12 year-old school children. The study comprised 86 children from three “control schools” and 102 children from four schools that represented the “intervention group,” which received “test meals” characterized by enhanced portions of fruits and vegetables. Dietary and anthropometric measurements were taken on all children (Lowitt et al., 2018).

FIGURE 1

Figure 1. Causal map showing the F2F theory of change, linking project interventions (red arrows) to anticipated outcomes (blue arrows). The ToC assumed a generally positive and linear impact pathway for improving nutrition outcomes. Double parallel lines indicate time delays in achieving anticipated outcomes.

Among the major outcomes of the F2F project were: (1) a significant increase in the consumption of vegetables and micronutrients by children offered the test meals; (2) through recruitment of a procurement coordinator by the project, the SMC was able to markedly increase its procurement of nutritious food items from local farmers; (3) benefitting from technology training, subsidies on agricultural inputs and professional support from the project, farmers were able to substantially increase crop productivity and diversity. Details of the project's implementation and overall findings are available elsewhere (Lowitt et al., 2015b, 2018; Mumena et al., 2018). Despite evidence suggesting positive nutritional and livelihood impacts over the course of the F2F project (Lowitt et al., 2018), the NSVC approach to local school feeding was ultimately not institutionalized by actors along the value chain, suggesting limitations or “blind spots” within the project's original ToC.

Methods

Utilizing a one-day workshop format, we conducted a participatory process evaluation of the ToC underlying the design and outcomes of the original F2F project (Figure 1). The workshop included representatives from different groups with an identified role in developing a local school feeding value chain and was undertaken in St. Kitts in November 2018, 4 years after conclusion of the F2F project.

Tools for Process Evaluation

Process evaluation has been described as a tool to “assess the fidelity and quality of implementation, clarify causal mechanisms, and identify contextual factors associated with variations in outcomes” (Craig et al., 2008). In their guide to process evaluation, Moore et al. (2015) highlighted the important influence of “context,” defined as “anything external to the planned intervention that serves to limit or enhance planned outcomes.” Furthermore, the “complexity of interactions in a place-based context is likely to exacerbate already existing complexity of the intervention itself” (Craig et al., 2008). This suggests that understanding place-based context is key to conducting process evaluation, requiring an understanding of food system-actor interactions in the conceptualization, design, and implementation of intended interventions. In order to connect a system dynamics perspective with the original F2F project interventions, we developed a combined Theory of Change (ToC) and Group Model Building (GMB) approach to facilitate participatory process evaluation, recognizing their potential complementarity (Nicholson et al., 2020).

A theory of change describes how and why an initiative works (Weiss, 1995) and the set of assumptions that explain both the mini-steps that lead to a long-term goal and the connections between these activities and the outcomes of an intervention or program (Stein and Valters, 2012). Understood as the logic, and necessary activities, the ToC reveals fundamental, often implicit, beliefs and assumptions held by those who design the planned activity (typically content experts) to achieve expected outcomes. According to Mayne and Johnson (2015), theories of change describe “results chains” that show the linkages between the sequence of steps in getting to impact, by being explicit about the causal assumptions behind expected impacts. Theories of change are generally presented in diagrammatic form but can take any format as long as they aid the process of uncovering and developing the assumptions within the theory of change (Stein and Valters, 2012).

Group model building is a widely used technique to bring “context” (local) and “content” (academic) experts together to better understand behaviors that give rise to complex contemporary problems. It is a facilitated process whereby groups of informed participants process information around a defined problem or issue. The problem or issue of interest is usually visually displayed in a form called a “reference mode” which then serves as the starting point for a facilitator-led process. Participants go through a nominal group technique (NGT) which elicits ideas from all group members on actions needed to cause system-wide change toward the desired outcome. Participants then engage in information exchange and co-learning at the cognitive level in order to develop a “causal map” which explains visually in one diagram the behaviors that give rise to the problem under study. The analysis of the causal map is directed at finding the structural elements that are responsible for the observed problem behavior. Feedback loops, closed chains of cause and effect, are seen as the primary drivers of behavior. A positive feedback loop leads to escalation: an initial increase in any of the variables in the loop will lead to further increases (an initial decrease would lead to further decreases). A negative feedback loop on the other hand leads to balancing behavior: an initial increase in one variable leads to decrease until a goal or limit is reached. After analyzing the map structure, such negative feedback loops can be used to identify “leverage points” for maximum impact for change (McCardle-Keurentjes et al., 2018).

Design and Facilitation of the Workshop

Efforts were made to ensure that diverse interests and expertise were represented at the workshop and that high-quality data on the school food system would be generated (Vennix, 1999). Experience with the implementation and results of the F2F intervention in schools (Lowitt et al., 2018) suggested the need for workshop participants to pay special attention to issues of produce procurement and logistics, staff training and kitchen infrastructure, design of menus, and children acceptance and consumption of meals. These elements of the F2F value chain seem to have been the main drivers of changes in healthy eating among children. These drivers were largely beyond the “farm gate” and fell within the realm of school meals governance. For this reason, in the selection of workshop participants, emphasis was placed on engaging policy actors representing organizations that were central to the functioning of the SFP (Table 1).

TABLE 1

Table 1. Categories of workshop stakeholders and number of participants in each category.

The first step in stakeholder selection involved the generation of a “master list” of potential participants (n = 70); this included field personnel and researchers who were directly involved in the F2F project, government officials involved in the SFP, and stakeholder groups and other agencies relevant to the success of an NSVC. In the second step, the master list was reviewed by team members who had been part of the F2F project to ensure that key actors had not been inadvertently omitted. The farming community was represented by the leader of one of the two local farmer organizations involved in the F2F project. Workshop participants from the public sector included agricultural extension officers, dieticians/nutritionists, the Executive Chef and other staff responsible for the school meals programs in St. Kitts and Nevis. Members of civil society organizations were among the 37 final workshop participants, representing five different stakeholder groups (Table 1). All participants had knowledge of the SFP and the F2F project and were able to provide meaningful information for the research. There was no representation from the private sector nor parent and teacher organizations.

The workshop began with F2F project researchers providing participants with a brief overview of the design, implementation, and outcomes of the F2F project. This presentation established the importance of SFPs as a vehicle for addressing the specific nutrition problem related to rising rates of overweight and obesity among children in the Caribbean. Government officials then delivered presentations on the operations and significance of the school meals program in SKN and the differences in funding models for the SFPs on the two islands. Formal presentations were followed by questions and answers, discussions, and comments, all of which allowed for updating group knowledge exchange about school feeding activities, childhood nutrition, public health initiatives and institutional support for school feeding programs in SKN. This step in the process was considered necessary to familiarize all participants with any institutional or community-level changes that may have taken place after the end of the F2F project in 2014.

Subsequently, the GMB approach commenced with workshop facilitators presenting participants with a diagram of the F2F ToC (Figure 1). In plenary session, F2F project researchers guided workshop participants through the underlying logic of the F2F ToC. This was followed by the participant elicitation process, using the nominal group technique, as adapted in GMB practice (Van de Ven and Delbecq, 1972; Vennix, 1999). The elicitation process began by inviting participants to identify missing connections or links in the F2F ToC (Figure 1) that might explain the lack of institutional uptake. Each participant was asked to write down their ideas, which were then listed on flip chart boards (Vennix, 1999). After approximately two rounds of elicitation, the solicitation of ideas ended. At this point, participants were divided into three sub-groups of 10–13 participants to optimize group engagement and facilitate the reflective exercise (Vennix, 1999). Each subgroup was asked to focus on one of the three main components of the F2F ToC (technologies for food production; procurement of locally grown produce for school meals; preparation of safe and nutritious lunch meals for children) and was provided with the list (“NGT list”) of the ideas previously generated.

A facilitator and a rapporteur guided each of the subgroups through the process of identifying, expanding, and connecting existing and new links and feedback loops in the F2F ToC toward the generation of an expanded causal map. Members of each subgroup selected ideas from the NGT list and explained to each other how proposed missing links were connected within the ToC diagram. During the process, facilitators challenged sub-group members to justify and verify proposed changes to the F2F ToC. When there was sustained disagreement within the subgroup, the facilitator would recommend that the item be temporarily set aside to allow discussions to continue. Through this facilitator-led process, workshop participants reviewed, evaluated, and revamped the F2F ToC. Stakeholders developed a shared understanding of the challenges to promoting and achieving healthy eating among children, and jointly produced a causal map. In the final stage of the process, participants discussed potential interventions and leverage points on the causal map that could help to institutionalize local NSVC in schools.

Data Collection and Analysis

Prior to collection of data, all research protocols were reviewed and approved by the UWI Mona Ethics Board (# ECP 15, 17/18), McGill University Research Ethics Board (# 231-1018) and National Ethics Board of St. Kitts and Nevis (# IERC-2018-06-013). Qualitative data analysis was conducted using manifest content analysis techniques (Morgan, 1993). Codes and themes generated from the qualitative data were systematically applied throughout the analysis (Sandelowski, 2000). This involved reflective and iterative review of transcribed group discussions that took place at the GMB workshop. Review of the data focused on emergent themes rather than using predefined or rigid categories. Throughout the data collection process, ongoing memo-development and writing activities, as well as peer debriefing and review, were employed as techniques to ensure the accuracy and reliability of the themes.

Results

While the original F2F intervention logic resulted in measurable successes [i.e., farmers were able to increase the quantity and diversity of local produce for school meals, and the acceptance by children of the newly designed nutritious meals exceeded 80% (Lowitt et al., 2018)], other factors, including a lack of school nutrition policies and cross-sectoral collaboration in school meals governance, and the need for substantial financial and professional resources to coordinate produce procurement from local farmers (Lowitt et al., 2018), may have stymied policy and practical uptake in the 4 years since the F2F project ended.

Overall, workshop participants supported the logic of the F2F ToC and viewed it as a critical part of a positive impact pathway toward needed changes in production, procurement and access to, and consumption of nutritious foods in schools. During their presentations, administrators of the SFP in St. Kitts confirmed that, since the conclusion of the F2F project, there had been no institutional or policy changes to the SFP. When presented with the F2F ToC, workshop stakeholders agreed that the F2F ToC model was novel and challenged traditional “silo” approaches to actions around school feeding. The following is a comment by one of the workshop participants on the scope of work and effort that would be needed to sustain the initiative: “[Behavior] change takes a long time, but we can do it incrementally… we can target every sector at the same time with different messages with the experts in those areas focusing on their particular peer group.” The F2F project was also viewed as promoting new exchange of information among policy actors, which fostered collaboration across sectors and connected government decision-makers in agriculture, health, and education with communities, school lunch service personnel and local farmers. Stakeholders acknowledged the attempt at NSVC development as having no precedent in SKN.

While the workshop participants (mainly context-experts) generally accepted the three fundamental components of the F2F ToC (production, procurement, food service delivery), they identified additional interacting components as “gaps” related to consumer education and health promotion; community engagement and social networking; public policy integration and advocacy; knowledge dissemination for capacity building; strengthened governance for domestic food production; market development and food pricing; and the need to address conflicting stakeholder interests. Figure 2 incorporates these components into a revamped ToC (referred to as RToC), leading to the development of a causal map that reveals a more complex theory of change. In building the RToC, stakeholders identified underlying impact and mediator pathways, new ways of thinking and engagement, and captured additional value chain activities needed to build a sustainable NSVC for school feeding in Saint Kitts.

FIGURE 2

Figure 2. Stakeholder-driven causal map showing a revamped and expanded theory of change (RToC), linking ten newly proposed interventions (green arrows) to F2F interventions (red arrows) and to anticipated outcomes (blue arrows) of the RToC. Double parallel lines indicate time delays in achieving anticipated outcomes. New interventions are placed in a black box, while all other proposed interventions build on existing local programs. FBO, faith-based organizations; SM, School Meals Centre.

We can identify in the causal map (Figure 2) three positive feedback loops that may drive improvement of nutrition outcomes for children in SKN. The first loop, on the left-hand side of the model, shows how positive changes in food purchasing behavior toward fresh local produce improves dietary behavior of school children, which in turn leads to increased micronutrient and macronutrient intake and, with time, improved growth and health status of children. The second loop, in the middle of the figure, shows the positive impact of improved availability of locally produced fresh fruits and vegetables delivered to schools on reliable produce collection and delivery service (and vice versa). The third feedback loop is visible in the right-hand side of Figure 2. This loop consists of the following relations: adaptive capacity of the rural community decreases vulnerability of smallholder farmers to climate change shocks, which in turn improves smallholder livelihoods. All three loops are positive, meaning that an initial increase in any of the variables in the loops will be reinforced further.

After stakeholders created the causal map (Figure 2) through the GMB deliberative processes, they were tasked with identifying intervention points to improve healthy eating among children. Table 2 shows the extended range and types of interventions and the associated impact pathways that participants identified, based on a shared understanding of how the multiple interacting components would need to be integrated across sectors, with multiple targets, for greater institutional uptake of a F2F school feeding model. Participants identified that targeting children through social media campaigns and technology-based games on healthy eating and improving nutrition awareness could be positive impact pathways in building a sustainable NSVC for school feeding (Figure 2, #1). This could be supported by a broader media promotion intervention for healthy eating (Figure 2, #9): “[A] strong education and marketing campaign for healthy eating in the Federation of St. Kitts and Nevis... should be supported by a strong policy environment that target key areas. … educate persons at multiple levels. … we want to target persons at multiple levels... parents… vendors… schools...the kids themselves.” Participants also highlighted the need for: promotional campaigns on healthy eating for new parents (Table 2, #8); community-based demonstrations on how to prepare low-cost nutritious meals using local produce (Table 2, #5); and the need for an overall school feeding policy (Table 2, #10) to support the objectives of the original F2F ToC. Encouraging school gardens and backyard gardening in communities was identified as another way to increase supply of local produce through the mediator pathway of “own production” (Table 2, # 3 and 4). Facilitating market access was considered an important impact pathway for strengthening food production by local farmers (Table 2, # 6).

TABLE 2

Table 2. Interventions identified by workshop stakeholders showing links to existing local programs and the relevant value chain actors.

Of the ten interventions identified, only three could be considered new: #5- Diversified menus and cooking demonstrations for parents and communities; #9- National promotion of healthy eating; #10- Integrated public policy on healthy eating. The other seven proposed interventions either adapted or built on existing programs to enhance the desired outcome. For example, in the case of proposed intervention #1, participants described a national youth development award program, designated “25 Remarkable Teens,” that they felt could be leveraged to engage “youth influencers” as health promoters in primary schools. Additional impact pathways, originally missing from the F2F ToC, became apparent and many of these pathways have also been noted in research on NSVCs (de la Pena et al., 2018). Participants also identified strengthening the infrastructure for school gardens as an important intervention strategy to increase the supply of fresh and nutritious produce for school meals, while also introducing food and nutrition education into the school curriculum. In strengthening the infrastructure for school gardens, however, there is a trade-off to be made; as school gardens meet the produce demands of the school meals program, the market opportunities for local farmers may decline, thereby dis-incentivizing farmers from participating in the school meal value chain.

Another insight that emerged during the GMB process was that the designers of the original F2F ToC may have underestimated the lack of consumer awareness of the role that school meals can play in supporting rural livelihoods and improving nutrition outcomes for children. It was in this context that a “re-branding” of the school feeding program and integration of public policy were identified as potential leverage points and impact pathways to improve healthy eating. When identifying the complexity of unhealthy eating in schools, policy actors in the workshop acknowledged the need for more integrated planning and new types of interaction across sectors and actors in the local food system: “[We] sat around this table … representatives from many sectors.. agriculturalists… health promotion… medical doctor, nutritionists and …we looked at a multi-problem approach to this issue (by asking the question). How can we then get everybody on board who are able to make this thing work?” This need for “subsidiarity” [defined as “decision-making at the level closest to the resource as possible … that fosters the conditions for implementation of management decisions” (Cosens et al., 2017)] became clear during group deliberation.

Discussion

Recent research by Guariguata et al. (2020) has shown that the GMB process can be an effective tool for engaging a diverse group of stakeholders in reaching a better collective understanding of the complex problem of unhealthy eating in the Caribbean, and for identifying potential interventions to address the problem. Building on this finding and integrating a theory of change approach into the GMB process, we engaged a diverse group of participants to deliberate and evaluate the design of the F2F project interventions in SKN. Our study aligns with recent calls for complex system approaches to better account for “real world” complexity in process evaluation, with evaluators articulating and mapping the theory of change, and then usefully building out the static system to generate hypotheses, before focusing on the key system dynamics affecting change (McGill et al., 2020). A theory of change approach, guided by GMB elicitation processes, enabled workshop participants to collaboratively appreciate different levels of complexity (detail and dynamic) by exploring the mental models underlying the interconnected system-wide issues influencing healthy eating by school children (Brownell, 2015; Amed et al., 2016; Whelan et al., 2018). In so doing, they also identified the local actors who would need to get involved in addressing the issue (Vennix et al., 1992; Anderson, 2004). The GMB process also revealed differences in the understanding of the system by local stakeholders (context experts) compared to researchers and academics (content experts). According to Van Kerkhoff and Lebel (2006), this dichotomy derives from the orientation of the researcher, who often generates theories devoid of local context, unlike stakeholders who are often embedded in that context.

For example, the original F2F ToC aimed to foster collaboration across local sectors and actors, by explicitly connecting researchers with farmers, SFP staff, and government officials in agriculture, health, and education. Through its internal logic, the F2F ToC acknowledged that the existing system needed to engage in new activities, rules, practices and knowledge exchanges. However, the GMB process revealed that the F2F project designers assumed that the additional project demands on local actors would be integrated into their existing work plans and practices in developing the local NSVC for school feeding. In practice, operationalizing the newly designed NSVC required greater institutional commitment and increases in cost, time, effort, and interactions among a wide range of value chain actors that could not be sustained (Sidaner et al., 2013; Sonnino et al., 2014; Jekums, 2015; Lowitt et al., 2018). Workshop participants displayed a shared understanding of the complexity underlying the local food system, the heterogeneity of interests among civil society, farmers, public sector actors and other groups, and advocated for additional intervention strategies that go well-beyond the scope of the original F2F ToC. These context experts, from various domains and sectors, were able to identify and explain to each other how the missing elements connected to, and would have improved, the original F2F ToC. This should be considered noteworthy, especially considering that existing public policy and food environment culture did not provide the necessary supporting conditions for building a sustainable NSVC in schools. In the absence of healthy school meals and school food vending policies to formally direct public resources and action, the effectiveness of the F2F ToC was limited because its needs and requirements had not been formerly identified as a national or local priority.

By combining both context and content experts in the development of the RToC through the structured process of the ToC-GMB workshop, the multi-level factors that interact to contribute to unhealthy eating among children could be assessed and discussed. Such a stakeholder-driven process has intrinsic value in that it builds legitimacy (Vennix et al., 1997) among local actors and end-users, thereby facilitating a place-specific dialogue that addresses different mental models and encourages dynamic hypotheses framed around questions such as, “how do we change this?” (Martinez and Richardson, 2001, p. 9). Our participatory approach to evaluation is consistent with the view of Ansah et al. (2018) who suggested that, through system dynamics approaches, participants engage in contextualized discussions that allow them to identify missing areas or gaps more easily for intervention and policy action. This can serve to refine and enhance the development of ToCs in future project design and post-project evaluation stages, thereby enabling system dynamics to be discussed before identifying hypotheses. Future experimentation and development of this approach in different food value chain intervention contexts are warranted.

Deliberative processes have been found to promote knowledge translation and exchange essential to evidence-based system-level evaluation, and decision making in complex systems (Boyko et al., 2014). In contrast to the original F2F ToC, which was developed primarily by content experts, the causal map arising from the GMB process involved a deliberative approach with both content and context experts working collaboratively in tasks of co-design and participatory evaluation. The benefit of this process was the inclusion of a broad base of ideas and the sharing of perspectives from key value chain actors. However, the challenge for this process, especially in the absence of national policy, is that the range and scope of proposed interventions may not be viable during a project term, and may be difficult to coordinate; furthermore, costs associated with long-term and complex projects may limit any sort of intervention at all. Nevertheless, the causal map (Figure 2) depicted the complexity of the food system and filled “intervention gaps” not addressed by the original F2F ToC. A benefit-cost economic analysis could be helpful in determining the viability and sustainability of new initiatives but failure to fully leverage the F2F initiative against existing national and community initiatives may also help to explain why, despite positive results, the F2F project interventions and outcomes were not institutionalized and sustained.

Strengths and Limitations

This study makes an important contribution to knowledge related to building and evaluating NSVCs in the Caribbean, an understudied geographical region of the world. Novelty in the study also lies in its adoption of a combined TOC-GMB approach to participatory process evaluation. According to de la Pena et al. (2018), a value chain approach offers great potential to unpack the complexity of food systems and identify entry points and pathways for strengthening the impact on nutrition. The authors argue, however, that success in building NSVCs depends on new and/or strengthened social networks, collaborative partnerships between public and private sector actors, and the expectation that proven interventions would subsequently become institutionalized in organizational, national, or regional policies. In our process evaluation research, the interventions proposed by expert participants generally align with the entry points identified by de la Pena et al. (2018) as being necessary for successful development of NSVCs. However, an important omission from the both the F2F ToC and the ToC-GMB process, and therefore a limitation of the research, was the lack of representation from the private sector; such representation could have made important contributions to understanding the barriers to building and institutionalizing a local NSVC. This is a critical knowledge gap since it was reported by workshop participants that up to 80% of the food procured by the school meals in St. Kitts and Nevis is procured from the private sector (supermarkets, food importers, wholesalers, food distributors). The study has two other main limitations. Over representation of public sector actors, under-representation of farmers, and lack of representation of parent and teacher organizations in the GMB process mean that actions on lessons learned from the GMB workshop may not necessarily lead to sustained availability and consumption of healthy meals in schools. For example, the sale of unhealthy food items in and around schools is an impediment to healthy eating among school children; without engagement of parents and vendors, efforts to create a healthy school food environment could be stymied. The other limitation of the study is the lack of economic analyses and evaluation of “system trade-offs” for procuring healthy but more costly food items for school meals and the implications of removing subsidies to farmers for agricultural inputs. While the mapping workshop identified three feedback loops driving improvement of nutrition outcomes for children in SKN, further research is needed to determine their relative impact on system behavior.

Conclusion

System dynamics thinking has previously been used in evaluation research (McIntosh et al., 2015; Allender et al., 2016); however, it has not been investigated or applied widely in the Caribbean or other Small Island Developing States, and especially not in combination with a theory of change approach to building and evaluating NSVCs. By combining a theory of change approach with group model building, our research incorporated complex systems thinking and led researchers and stakeholders to develop shared understandings of how intervention-related outcomes occur within the school food system. The results demonstrate the value of utilizing group model building and elicitation processes to engage local and regional stakeholders in the design of projects addressing complex problems. Workshop participants were able to develop a shared understanding of the importance of local NSVCs for school feeding and the need to craft a system-based response to the growing problems of unhealthy eating and obesity among children in the Caribbean. Participants also identified a wide range of interventions and associated assets in the existing food system that could be drawn upon to promote and sustain healthy eating in schools. Efforts to develop sustainable NSVCs through the existing SFPs operating in the Caribbean will require the ongoing engagement and education of diverse communities to reconcile differing understandings while meeting the challenges of resource limitations and conflicting stakeholder interests. The rapid environmental changes and economic transformations occurring globally place Small Island Developing States, like St. Kitts and Nevis, at the frontline of efforts to address complex food system challenges (Connell et al., 2020; Hickey and Unwin, 2020; Thomas and Theokritoff, 2021), requiring shared understanding, system-wide policy and institutional adjustments and increased roles for civil society when identifying and designing local food system interventions.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by McGill University Research Ethics Board (# 231-1018). National Ethics Board of St. Kitts and Nevis (# IERC-2018-06-013). The participants provided their written informed consent to participate in this study.

Author Contributions

ASV and GH: conceptualization. ASV, ER, GH, and LG: methodology. ASV, ER, LG, GH, and NU: formal analysis. GH and LP: resources. ASV, GH, and LP: writing and original draft preparation. AS, GH, and NU: funding acquisition. All authors: writing—review and editing. All authors have read and agree to the published version of the manuscript.

Funding

This work was carried out with the aid of a grant from Canada's International Development Research Centre (IDRC), Ottawa, ON, Canada (grant number ID 108690). The funding source was not involved in the conduct of this research which was completed as part of a larger project entitled: Improving Household Nutrition Security and Public Health in the CARICOM (FaN- Food and Nutrition).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We are grateful to Mrs. Latoya Matthew-Duncan, Country Team Lead for St. Kitts and Nevis, for providing logistical support in the field. Sincere thanks to the research participants who contributed their valuable time, insights and knowledge.

References

Allen, S., and de Brauw, A. (2018). Nutrition sensitive value chains: Theory, progress, and open questions. Global Food Sec. 16, 22–28. doi: 10.1016/j.gfs.2017.07.002