Crowdsourcing priorities: a new participatory ex-ante framework for crop improvement

Demand-led approaches to crop breeding involve ranking priorities across different disciplines and stakeholder categories, but the implications of decisions made during varietal development are frequently understood only years later. Breeding teams must work a priori to rank crop improvement priorities and product concepts considering the context of the current, and ideally future, environmental, production and market conditions that a variety will be entering upon release. We propose PEEP (Participatory Ex-antE framework for Plant breeding), a new ex-ante framework, as a methodological tool for priority setting in plant breeding. PEEP leverages two elements: the usage of a heterodox methodological approach and the strong emphasis on the participation of knowledge-rich stakeholders. PEEP ranks crop improvement impacts based on a heterogenous set of environmental, social, and economic benefits and it employs a recursive and tailored multi-stakeholder approach to relate crop improvement impacts and product concepts. PEEP builds on the need to engage technical as well as practical knowledge and utilizes a tailored engagement strategy for each knowledge-rich stakeholder involved. The outcome is an assessment that ranks crop improvement impacts and breeding product concepts according to designed set of criteria. PEEP is scalable, gender inclusive, and crop agnostic. The results of PEEP are ex-ante recommendations for breeding teams in National Agriculture Research centers (NARs) and CGIAR centers alike. This methods manuscript describes the theoretical foundations of PEEP and its four phases of implementation.

1. Introduction

Crop breeding is a unique field in which the implications and impacts of decisions made during varietal development will be understood only years later when the resulting variety is released to farmers. Breeders make decisions a priori that consider both current and, ideally future, environmental, production and market conditions into which a variety will be released. Significant shifts in breeding paradigms, under the banner of “modernization,” now position demand or market-led approaches to be non-negotiable (Tarjem et al., 2022). Yet this reorientation and need to respond to complex diversity are at odds with the limited resources most public sector crop breeding for development programs possess. This creates a need for research prioritization within crop breeding programs (Pemsl et al., 2022). A growing body of literature documenting how social differences drive trait and varietal preferences complicates the picture (Fisher and Carr, 2015; Weltzien et al., 2019), asking breeders to understand the priorities that women and men assign to genetically determined traits (Orr et al., 2018). Moreover, demand-led approaches necessitate ranking priorities with an interdisciplinary team, merging needs from different disciplines and experts, such as plant breeders, gender specialists, rural development experts, agricultural economists, and value chain stakeholders (Pemsl et al., 2022). Current demand-led paradigms that focus on triangulating on-farm genetic gains, market responsiveness, and social impacts need priority setting frameworks that link crop improvement priorities, preferred traits and expected impacts. In this methods manuscript, we describe a participatory ex-ante framework for plant breeding priority setting, focusing on crop improvement priorities¹ and product concepts. The framework, called PEEP (Participatory Ex-antE framework for Plant breeding), is developed to relate crop improvement priorities, expected impacts and hypothetical new varieties. The framework centers on the question: when targeting a specific impact, which hypothetical new variety (expressed in the form of a product concept)² should be prioritized by breeding programs? Complementing existing participatory breeding approaches, PEEP explores “why” a breeding priority is important and most impactful, in addition to “what” crop improvement priority is top-ranked.

PEEP leverages two tenets to answer this question: a heterodox methodological approach and the engagement of knowledge-rich stakeholders. In contrast to ex-ante frameworks built on economic surplus and optimization modeling, PEEP ranks priorities based on a heterogenous set of environmental, social, and economic impacts determined by stakeholders. Furthermore, PEEP employs an iterative and tailored multi-actor approach. Building on the principle of engaging technical as well as practical knowledge, PEEP involves an array of knowledge-rich stakeholders. The outcome is an assessment that ranks crop improvement priorities and breeding product concepts according to a designed set of criteria, co-created with stakeholders and breeding programs. PEEP produces ex-ante recommendations for breeding teams in National Agriculture Research Centers (NARs) and other crop improvement research centers, including CGIAR, using an analysis which is scalable, gender inclusive, and crop agnostic (Mills, 1997). Complementarily, PEEP could also function as a monitoring tool, to align the research agenda of breeding programs to both existing high-level objectives (e.g., 2030 Sustainable Development Goals) and local stakeholders’ needs.

Below we describe the theoretical foundations of the framework and its four phases. In section 2, we locate the work in the literature and highlight the novelty of the framework. In section 3 the four phases of the PEEP framework are described. A brief discussion (section 4) on advantages and limitations of PEEP concludes the manuscript. A pilot with the Institute of Environment and Agricultural Research (INERA) in Burkina Faso is underway to test the practicability and reflect on the experience of PEEP, and those results will be published separately.

2. Background

2.1. Situating PEEP in ex-ante priority frameworks

Crop breeding is a discipline grounded in foresight of future needs of growers, processors and consumers. Setting research and development priorities ex-ante is therefore a necessity to succeed in meeting these demands. We examined the literature around ex-ante research priority setting approaches to situate our work and identify gaps and opportunities for methodological development. To date, different approaches have been developed to support priority setting in international agricultural research programs (Wiebe et al., 2021; Alston et al., 2022). Applications span from CGIAR-level (case studies can be found in Raitzer and Kelley, 2008), to national research prioritization (e.g., EMBRAPA in Brazil as described by Avila et al., 2002). Recently, the use of international agricultural research prioritization exercises has been more sporadic, with less data intensive procedures preferred (Thornton et al., 2018). These alternative methods range from simple qualitative scoring exercises to highly complex simulation models estimating the functional relationship between inputs (research investments) and agricultural outputs while accounting for the underlying uncertainty (see Braunschweig (2000), for an overview of different priority setting methods).

We summarize existing ex-ante priority setting frameworks (Supplementary Table A1) and characterize them along three dimensions of scale of analysis, translation of benefits into dollar values, and sex disaggregation. These three dimensions represent junctions at which ex ante frameworks that are both locally relevant and gender responsive distinguish themselves from more traditional approaches. Most existing frameworks focus on benefits in economic terms, and rarely using sex-disaggregated data (Supplementary Table A1). Country-level assessments are preferred, even though frameworks allowing a flexible scale of analysis (i.e., national and regional) are common.

Guided by an interest to develop a methodology that could be more participatory and engaging for respondents, we further looked at frameworks for their level of participation and engagement with stakeholders. Participation describes the degree at which studies involve a variety of different stakeholders, beyond scientists. Engagement exemplifies the frequency at which stakeholders are involved and if feedback and validation mechanisms are put in place. Studies in the early 1990s guided by economic surplus theory and cost benefit analysis are less participatory and engaging (for a review see Braunschweig, 2000). Recent modeling approaches with heterogeneous agents are also not participatory or engaging (e.g., Endresen et al., 2011; Groot et al., 2012 in Supplementary Table A1). Participatory varietal selection (PVS)-like approaches where alternative research options are validated by non-academic stakeholders (mostly farmers) are more participatory, but the engagement of stakeholders occurs only as an a-posteriori consultation with no feedback loop, where priorities from stakeholders are then validated jointly with scientists (e.g., Randolph et al., 2001; Pemsl et al., 2022 in Supplementary Table A1). Crowdsourcing plant breeding methods (e.g., Steinke and van Etten, 2017 in Supplementary Table A1) engage stakeholders in an iterative manner with easy to implement approaches but they include primarily farmers. A rare example of a highly participatory and engaging framework was Blundo‐Canto et al. (2020) in Supplementary Table A1, but also highly complex and abstract.

Methodologies currently available for ex-ante research priority setting encompass simple interactive scoring exercises (e.g., participatory ranking scenarios) to complex simulation models estimating the functional relationship between inputs and outputs (e.g., agent-based models; Supplementary Table A1). Allocating research efficiency and selecting the most promising research activities are issues directly tied to the scarcity of resources for plant breeding in development. Therefore, most existing frameworks place an emphasis on economic efficiency and on costs and benefits that can be expressed in monetary values (Braunschweig, 2000). The economic surplus analysis and the cost–benefit analysis are the tools most frequently utilized. Despite being easy to interpret, these two techniques present a few methodological disadvantages. First, they rarely include non-quantifiable and non-marketable outcomes (e.g., the shadow price of gender-related benefits). Second, in these tools, agricultural researchers provide most of the input, and active participation of stakeholders is quite limited. Third, these tools offer a static representation of the commodity market and thus tend to underestimate longitudinal non-linear dynamics which can affect the breeding process (Petsakos et al., 2018). For example, although some implementations allow for an explicit representation of dynamics in production and consumption (HarvestChoice, 1995), economic surplus models able to analyze well-structured foresight scenarios (like those proposed by the Intergovernmental Panel on Climate Change (IPCC) Davis et al., 1987), are frequently complex and data intensive. Lastly, this economic surplus paradigm has raised concerns because externalities, distributional effects, and longer-term impacts all tend to be neglected with a narrow focus on breeding costs and benefits (Dahlberg, 1988).

2.2. What’s new? Novelty of PEEP

To develop PEEP, we took inspiration from the systematic, quantitative ex-ante priority assessment undertaken by the CGIAR Research Program on Roots, Tubers and Bananas (RTB) in the period 2012–2014 (Pemsl et al., 2022). PEEP preserves the systematic and quantitative nature of this assessment, while integrating attention to gender and adapting for utility for national level and crop-specific breeding projects. PEEP complements the literature on methods for ex-ante priority assessment in crop breeding through three methodological advancements. First, PEEP is adaptable at the national level, building on the existing work (Mills and Karanja, 1997; Randolph et al., 2001; Singh et al., 2020; Pemsl et al., 2022). Unlike these studies, PEEP leverages the national focus to better consider the relevance of gendered preferences and access to resources, which are context dependent. Second, PEEP is designed to be consultative in all phases, resembling the frameworks of Pemsl et al. (2022) and Steinke and Van Etten (2017). PEEP includes an array of stakeholders beyond scientists, so that crop improvement priorities are set and validated by social and natural scientists as well as practitioners. Here we define practitioners as local actors directly engaged in agricultural production, processing, and marketing and representatives from, e.g., seed companies, agricultural cooperatives, agricultural women’s groups, and national or regional policy makers. PEEP is purposively combining multiple participatory tools to ensure that each category of stakeholders is involved through an approach which maximizes participation and engagement. Third, PEEP considers gender dynamics³ by seeking gender equity in the evaluation of alternative crop improvement priorities, and product concepts. To the best of our knowledge, no existing ex-ante framework thus far deliberately includes gender as a lens of analysis in prioritization.

Each of these three methodological innovations of PEEP has challenges. For example, PEEP requires a heterogenous group of stakeholders with deep contextual knowledge and local impact pathways. This may sometimes lead to conflict between national and local priorities. Establishing a process to reconcile these conflicts is a key component of the PEEP framework. Furthermore, it is particularly challenging to capture the views of the most vulnerable, but often least accessible, populations. PEEP includes nationally representative stakeholders who have an overview of the relative importance of different opportunities and barriers faced by these populations, but a risk remains that the needs of the most vulnerable respondents may still be missed. Finally, integrating a diversity of stakeholders and gender analysis requires additional expertise and creates a further level of complexity. Trade-offs hinge on who might lose or benefit from new varieties.

We leverage the use of heterodox methodological approaches and a strong emphasis on the engagement of knowledge-rich stakeholders to circumvent these challenges. PEEP ranks priorities based on a heterogenous set of benefits without quantifying impacts in terms of economic return (or net present value). Furthermore, in contrast with other scientific domains (e.g., health and medicine) where priority setting and ex-ante analysis are well-established practices and where engagement is actuated through the Delphi method (Linstone and Turoff, 1975), we utilize a less common iterative and tailored multi-stakeholder approach. Built on the principle of engaging technical as well as practice knowledge, PEEP adapts a diverse tool of engagement to each category of knowledge-rich stakeholders. This avoids multiple, highly resourced, and time intensive, rounds of consultation, especially where participants are not familiar with repetitive research-oriented routines.

How is PEEP any different from existing participatory plant breeding efforts? PEEP is interested in the “why” more than in the “what.” Linking crop improvement impacts to product concepts, PEEP generates ex-ante impact pathways. These support breeders in justifying the impact of a future variety when the variety is still simply a product concept. Having clarity on which product concept targets the set of crop improvement priorities considered relevant by a heterogenous group of stakeholders assists breeders in (i) justifying the impact of new varieties under development; (ii) ensuring that product concepts target priorities and impacts are equitable and fair (iii) justifying the investment made by national and international funders; and, finally, assists breeders in (iv) better marketing of new varieties.

3. Description of the proposed method

PEEP has four sequential phases (Figure 1), detailed in this section.

FIGURE 1

Figure 1. Map of the four sequential phases of PEEP.

3.1. Phase 1: creation of core implementing team

In the first phase, the breeding program implementing PEEP must form a core implementing team (hereafter core team). The core team oversees the process development, analyzes the results, and compiles the final assessment report. A set of competencies underlie the selection of individuals for the core team (Box 1) to ensure that the right balance of skills and experience are represented on the team are driving this process. Once assembled, the core team decides the geographical scope of the application (national vs. regional) as well as the target crop. PEEP can accommodate evaluations at the country level or regional level on any crop product concept to be evaluated.

BOX 1. Set of competencies for the core team

Competencies common to all core team members

Values diversity of perspectives and experience

Seeks representation of social science in research teams and fosters interdisciplinary dialogue

Builds a supportive culture within the working group

Values academic as well as practical knowledge and fosters the exchange of knowledge among experts and stakeholders

Bridges research and development practice

Is open to continuous improvement as a method for improving the research process and its effectiveness

Competencies for breeders, geneticists, pathologists, entomologists, and other members of the breeding team

Demonstrates scientific rigor in the breeding subject of competence

Contributes to breeding scheme design

Contributes actively to developing or improving crop product concepts for the chosen crop in country

Engaged with participatory varietal selection, or participatory plant breeding more broadly

Engaged in the breeding team of the organization and contributes to breeding targets set by the organization

Competencies for gender specialists

Proven foundational gender analysis competencies

Conducts high quality gender research

Interprets and communicates the implications of gender relations as well as gender differences to a multi-disciplinary team at different stages in the research cycle to help the team identify constraints, opportunities, priorities, research outcomes, impacts that need to consider gender

Demonstrate ability to propose and lead a scientific research project addressing social and gender issues

Produces research on social and gender issues suitable for publication

Applies advanced social science concepts and knows how to deepen analysis beyond simple sex-disaggregated comparisons to define implications or outcomes of gender inequality

Makes skillful use of advanced social research design, data collection and analysis techniques to conduct research on strategic gender issues

Leads and champions greater understanding of the relevance of gender to agricultural research

Competencies for climate specialists

Works with climate predictions and meteorological data

Engaged in the use of meteorological data for agriculture

Proposes and conducts research on climate and agriculture

Advocates for including climate in interdisciplinary debates

Integrates climate-smart approaches in plant breeding

Competencies for agricultural economists

Fundamental knowledge of agricultural marketing and economics

Independently conduct of assists researchers in performing economic analysis on a variety of issues related to the agriculture sector

Translates data into written reports/economic analysis, connecting results of analysis to actionable information

Understands how language and culture shape meaning in socioeconomic data collection

Understands basic principles of sampling and controlled comparison in data collection

Has knowledge and experience of different ways to deliver products and services effectively to rural population, to define realistic goals and measurable impacts

Has the skills to independently collect or supervise the collection of reliable social-disaggregated data

Has knowledge and direct experience of ex-post impact assessments, and possibly ex-ante

Has previously engaged actively with participatory varietal selection, or participatory plant breeding

Knowledge of plant product concepts and market segments is a plus, but it is not required

3.2. Phase 2: selection of knowledge-rich stakeholders to form PEEP stakeholders’ groups

Assembling a team of stakeholders to engage in the PEEP framework is the second - and possibly most critical - phase of the PEEP methodology. According to their degree of familiarity with the formal (academic) research process, stakeholders are assigned to two groups: (i) group R (Research), and (ii) group P (Practice). The Research group involves any member of a cross-functional breeding team who has technical knowledge of the target crop (see Box 2). Complementarily, the Practice group includes all stakeholders involved into the targeted crop value chain and possess practical knowledge (see Box 2). Engaging the R and P groups with transparency is important, clearly informing each group member about time and resources required to attend the process. For group R, incentives to participate in PEEP might be: (i) contribution to better define national or regional breeding objectives, (ii) the possibility to prioritize gender and climate in the national breeding agenda, (iii) better allocation of resources to crop breeding programs, (iv) possibility to conduct an evaluation which attracts international donors and funding agencies, and (v) networking and round table opportunity for new projects. For group P, incentives might be: (i) steering the breeding work to account for their needs and preferences, (ii) contribute to the development of varieties which are better suited for production and selling purposes, (iii) build social networks with researchers, (iv) tighter links with the local research community, which yield learning opportunities and higher engagement.

BOX 2. List of categories in group P and R

Group R (research)

Any member of a cross-functional breeding team, including, e.g.:

Breeders and Geneticists*

Agronomists, pathologists, entomologists, and other members of the breeding team*

Agricultural economists (with expertise in local markets and prices, formal and informal seed systems)*

Gender specialists*

Extensionists, including some with focus on gender

Nutrition specialists (if available)

Climate experts (from National Meteorological Office)*

Food scientists / Food processing experts

Mechanization experts

State or regional policymakers with mandate on agriculture

National policymakers with mandate on agriculture

Non-profit organizations involved in agriculture, development, and gender

Donors – in country and regional missions

Group P (practice)

Producers (social heterogeneity represented, with equal voice to marginalized producers)*

Traders

Processors*

Consumers*

Seed companies

Representatives from farmers agricultural cooperatives and organizations

Representatives from informal and formal agricultural networks

Representatives from women’s agricultural groups*

Agro-input dealers

Regulatory bodies for certifying seeds and GM production

Consumers’ organizations - including, among others, representatives from women’s entrepreneurs’ groups and representatives from women’s advocate groups

*Non-negotiable members of each group to ensure representative and actionable results.

Selecting members for the Research group should include consideration of their expertise on the chosen crop, whether they are young researchers or senior leaders, as well as if they are regional or national collaborators working at in-country international centers.

For members of group P, choosing representatives at the national or regional level may be less straightforward. The core team needs to effectively sample a sub-population of producers, traders, processors, consumers, formal and informal agricultural cooperatives, and women’s groups. Both at the national and regional level, NARs or CGIAR centers should utilize an informed stratified sampling strategy. If a list of crop producers, processors or traders for each area is available, we highly encourage the core team to use this information to calculate sampling weights

3.3. Phase 3: survey and choice experiment to identify crop improvement priorities

3.3.1. Survey

At the start of phase 3, relevant research priorities are elicited from group R through a large-scale expert survey conducted either online or in person. PEEP provides a generic structured questionnaire as guidance, but the core team adapts and tailors the survey to the crop of reference and specific national context so that it is most useful. Overall, the questions lead respondents to list and explain constraints related to breeding, economic, and gender, as well as climate issues. The questionnaire contains two sections: the first section includes open-ended questions common to all respondents, while the second section includes closed questions, tailored to each respondent’s competencies. In the second section, respondents are asked to rate the importance of different research priorities using a five-point scale (from “not important” to “very important”). Examples of research priorities are breeding for drought tolerance, improving seed storage, improving traditional processing techniques, developing new products for industrial application, reducing men’s and women’s health risks of on-farm insecticides use and many more. The first part of phase 3 closely resembles the second step of the RTB ex-ante framework described in Pemsl et al. (2022).

Research priorities are divided into thematic subsets. Each member of the group R will respond solely to the subset of priorities matching their domain of competence. This is done to avoid missing data or including biases in the ranking of research priorities. Data on the personal attributes of the respondents are collected in a third section of the survey.

The following thematic subsets with their appropriate respondent categories are exemplified below:

– Crop Improvement domain, whose research priorities are rated by

o Breeders and Geneticists

o Agronomists, pathologists, entomologists and other members of the breeding team

– Processing domain, whose research priorities are rated by

o Food scientists / Food processing experts

o Mechanization experts

– Marketing, whose research priorities are rated by

o Agricultural economists (with expertise in local markets and prices, formal and informal seed systems)

o Non-profit organizations involved in agriculture, development and gender

o State or regional level policymakers with mandate on agriculture

o National policymakers with mandate on agriculture

– Cross-cutting themes, whose research priorities are rated by

o Gender specialists

o Extensionists with focus on gender

o Nutrition specialists (if available)

o Non-profit organizations involved in agriculture, development and gender

o Adm2 policymakers (at the state or regional level) with mandate on agriculture

o National policymakers with mandate on agriculture

– Climate, whose research priorities are rated by

o Climate experts (from National Meteorological Office)

o National policymakers with mandate on agriculture

Data on the personal attributes of the respondents are collected in the third section of the survey.

Following the strategy in Pemsl et al. (2022), research priorities in the survey need to conform with the following criteria: (i) the research creates a global public good in the form of a new, adoptable product concept addressing a key constraint or targeting a PEEP opportunity for the crop of reference; (ii) impact would materialize within the 25-year assessment period; and (iii) the research scope is within the NAR or CGIAR center capacity and its mandate, prioritizing the needs of (smallholder) farmers and other vulnerable groups in the country. Furthermore, listed research priorities must be addressable in the next 5–10 years, given the technical and institutional capacity of the breeding programs involved. This helps the group to understand what research can be done, in addition to what is priority, and it gives a sense of possibility that can be acted on.⁴ Results from the survey are the first research output of PEEP and represent per se an interesting overview of how research priorities are listed and ranked among disciplines.

Overall scores within each thematic domain enable selection of research priorities. The first three⁵ top-ranked research priorities in each thematic domain are selected to be included in the choice experiment with group P (see below), for a total of 12 possible research priorities. The domain of crop improvement is excluded because priorities in those areas are used to develop the product concepts.⁶ The selection of product concepts should consider the scope of the NAR or CGIAR center research activities to ensure a good match of assessed options with the program portfolio. In addition, the core team reviews the final list of research priorities to summarize if and how they are considerate of gendered preferences.

Once the list of crop improvement priorities is ranked by the group R, the core team will develop a list of crop improvement impacts which are directly linked and derived from the priorities selected. For example, if the research priority is “Development of new cowpea varieties which are drought resistant,” the corresponding impact will be “Help you deal better with drought while cultivating cowpea.” This step ensures that priorities are intelligible for actors of group P and are seen as actionable.

3.3.2. Choice experiment

Once the list of rank crop improvement impacts is assembled, the group P is formally engaged into the PEEP framework. Group P is involved through a crowdsourcing method which closely mimics gamified choice experiments. Gamified choice experiments follow a strict and replicable guideline and provide quantitative results, which can be compared with previous evaluations and the results of alternative methods for priority setting (for an application in the domain of priority setting see the paper of Steinke and Van Etten, 2017).

The choices presented to respondents consist of a set of three product concepts, starting from available end-users’ preferences on crop traits and existing product profiles. Here we draw on recent approaches of concept testing to explore motivations behind the choice of one variety over another (Rutsaert et al., 2022). Concept testing entails showing a new product idea through a description or visual material, with the goal to obtain feedback and eventual interest in purchasing the variety. Product concepts are brief narrative descriptions, easy to interpret and present. They describe the morphological characteristics of the seed and plant variety, the main grower requirement that the variety addresses and conclude with an additional list of standardized information (e.g., yield potential, fertilizer needs, maturity, grain usage). Beyond containing the narrative for the product concept, the script includes suggestions on posture, voice tone and other non-verbal instructions to ensure a clear understanding of the product concepts by the stakeholders (Rutsaert et al., 2022). Product concepts are developed by the core team based on existing product profiles, and crop improvement priorities from the survey with group R.

Each product concept is then presented along with the research impacts derived from the survey with group R (see Figure 2). Respondents will be asked “Which of these three new varieties will help you the most to achieve impact A? Which will help you the least?.” The three new varieties, presented in the form of product concepts, will be assessed against all 12 impacts (Figure 3 provides an example). Respondents will have the opportunity to declare that the impact is linked with none of the three product concepts proposed (exit strategy). Enumerators will also present research impacts in a random order to each respondent, to ensure that not always the same impacts are presented at last when survey fatigue is at the highest peak (Figure 3).

FIGURE 2

Figure 2. Stylized representation of the choice experiment.

FIGURE 3

Figure 3. Example of the choice experiment.

Once the choice experiment is concluded, the core team utilizes well-established ranking models, especially the Plackett-Luce model, to analyze the data. Calculating the log-worth of each research impact, the core team is able identify which is the product concept with the highest probability of being associated with each impact by the category of group P interviewed.

3.4. Phase 4: validation and feedback

The main result of phase 3 is the prioritization of impacts from crop improvement impacts and how they map to a set of given product concepts. This helps breeders to align their prioritization of product concepts to meet impacts. During this phase, breeders, geneticists, agronomists, pathologists, and other members of the breeding team are responsible for integrating crop improvement impacts. Concretely this means answering two main questions: are existing product concepts able to address the impacts? If not, what information is missing?

A tension might emerge from this validation: on the one hand, the breeding team needs to work with product concepts that are realistic given the technical and institutional capabilities of the NAR or the CGIAR center. On the other hand, the team needs to ensure that previous efforts do not get lost in the name of “breedability” of alternatives.

We encourage the ex-ante team to repeat the analysis at every new breeding cycle. If the team perceives that a change in priority might have occurred earlier–due to a new agricultural policy, a climate event, or an expected shock like the Covid-19 pandemic–the ex-ante exercise can be repeated prior to the closing of the breeding cycle.

4. Discussion

We started this methodological work out of necessity. When working with NARS breeding programs to provide support on how to systematically, and most importantly, inclusively set crop breeding priorities, we were unable to identify methods to do so. When we scoured the literature on methods to set crop breeding priorities ex-ante, holistically and inclusively we identified a gap that this methodological work has attempted to fill. We outline in this paper PEEP, a method that provides an answer to a key question for breeding teams: when intending to achieve a specific impact, which product concept should be prioritized for development? To our knowledge there is no other method or approach that seeks to directly engage farmers, and other end users and beneficiaries of crop varieties, to link impacts that are important to them, with options for breeding programs to work on (Brown et al., 2020). In this sense PEEP creates the opportunity for a broader range of stakeholders to “have a say” in crop breeding priority setting. This consultative, if not participatory, underpinning to the work is timely and an important contribution to the field at a time when demand-led breeding is taking center stage. Just as participatory plant breeding can be described as highly client-oriented breeding (Witcombe et al., 2005), we see PEEP as highly client-oriented crop breeding priority setting. PEEP is therefore closer to the Blundo‐Canto et al. (2020) ex-ante prioritization framework.

Creating meaningful choices of crop varieties can be understood to be a form of empowerment for marginalized farmers (Polar et al., 2021). Using this framing, we place importance on engaging a broad range of socially heterogenous farmers, with emphasis on engaging marginalized women and young farmers in design and implementation of PEEP. This allows the framework to conform with the minimum standard of sex-disaggregation in data collection needed to conduct gender analysis (Doss and Kieran, 2014) and can be adjusted to add layers of socioeconomic data to further enhance analysis. In doing so, PEEP allows breeding programs to observe if targeted impacts are prioritized equally by different actors involved, or if a socially distinct sub-group (e.g., illiterate women or men widows or widowers, single parents) would give different weights to alternative impacts.

Departing from cost–benefit and investment considerations, PEEP leverages methodological approaches typical of heterodox economic disciplines. This entails relaxing assumptions around economic efficiency in favor of non-quantifiable and non-marketable outcomes, produced by rankings elicited from technical and non-technical actors endowed with different types of knowledge and power. In doing so, PEEP contributes to expand the field of priority setting beyond ex-ante investments allocations to include considerations on externalities, distributional effects, and longer-term impacts.

In building a flexible and versatile ex-ante priority setting framework, we aim to demonstrate that national and international research organizations would benefit from systematic and integrated priority assessment cycles that are repeated and constantly adjusted over time, with deliberate learning incorporated into each loop. This will strengthen frameworks and processes, contribute to institutional memory and capacity building, and increase relevance of priority setting for decision making while reducing its costs.

These insights are crucial for the public sector breeding for development, that distinguishes itself by explicitly focusing on social inclusion outcomes, such as gender equality, poverty alleviation and food security as laid out in the sustainable development goals. We expect the framework to guide public crop breeding institutions, such as national agricultural research centers and CGIAR centers. Complementarily, the framework has the potential to be appealing and benefit private breeding programs too. Understanding clients’ breeding priorities and expected impacts help private programs to create better products, with higher adoption, profits and return on investment (Ragot et al., 2018).

The framework has limitations. First, it does not account explicitly for trade-offs among crop improvement priorities and impacts. Trade-offs hinge on who might lose or benefit from the breeding process. For example, reduction in yield loss might matter to a farmer but market share by women purchasing a lower-yielding but lower-labor or higher-nutrient crop might matter more to a seed company if it increases sales among women. In aggregating results from the choice experiment, trade-offs and win-lose become less visible to the breeding team. Second, the framework does not plan for a validation step between the survey and the choice experiment: the group P can choose the best and worst combination between impacts and concepts, but, in the current version of the framework, they can neither expand nor modify the set of impacts on which to perform the choice. Piloting the framework will help to mitigate these pitfalls, while possibly highlighting others.

5. Conclusion

There is an increasing need for systematic priority setting to guide resource allocation in international public agricultural research. Effective research prioritization in crop breeding requires an ex-ante evaluation of program activities. The PEEP priority setting ex-ante framework proposes an interdisciplinary, multi-stakeholder and gender-intentional approach to rank crop improvement priorities, impacts and product concepts to perform an ex-ante breeding assessment. While methods and tools within PEEP are not new, taking them to scale and incorporating multiple objectives by analyzing gender, and national/regional benefit allocation represents a substantial advancement over previous efforts. We also experiment with ambitious targeting and stakeholder engagement processes, which help to ground truth the selection of research options, resulting in a high level of stakeholder awareness, and yielding potentially important lessons learned.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the [patients/participants OR patients/participants legal guardian/next of kin] was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

MO: Conceptualization, Formal analysis, Methodology, Visualization, Writing – original draft, Writing – review & editing. DR: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing. HT: Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing.

Funding

The authors declare financial support was received for the research, authorship, and/or publication of this article. We acknowledge the support and funding of the CGIAR Excellence in Breeding (EiB) Platform. This publication is made possible by the generous support of the American people through the United States Agency of International Development (USAID). The contents are the responsibility of the authors and do not necessarily reflect the views of USAID or the United States Government. Program activities are funded by the United States Agency for International Development (USAID) under Cooperative Agreement No. 7200AA-19LE-00005.

Acknowledgments

We thank deeply Eveline Compaore and the whole cowpea breeding team at INERA (Burkina Faso). Interactions with this team have shaped and improved the PEEP framework substantially. We are sincerely grateful that this team decided to partner with us in piloting PEEP. We thank David Brown (Cornell University, USA) for the support in better designing the PEEP choice experiment phase. We are indebted to Lora Forsythe (Greenwich University, United Kingdom), for her guidance on improving the gender inclusiveness and participatory nature of the framework. We thank Peter Coaldrake (CIMMYT) and Jason Donovan (CIMMYT) for their support and interest in this work. We thank the participants from various National Agricultural Research Centers and Universities, for providing their feedback on the framework during the 2022 Feed the Future Innovation Lab for Crop Improvement Annual Meeting (Saly, Senegal): in particular, we thank experts from KALRO (Kenya), Sokoine University (Tanzania), NaSARRI (Uganda), BAME (Senegal), INTA (Costa Rica) and Quisqueya University (Haiti). At last, we acknowledge Graham Thiele (RTB Program, CGIAR), Jacqueline Ashby, Berber Kramer (IFPRI), Jason Donovan (CIMMYT) and reviewer for their feedback on earlier versions of this manuscript.

Conflict of interest

DR is an owner of Cultural Practice, LLC. DR has no direct or indirect conflict of interest with Cornell University, Frontiers, or the organizations involved in or funded by this study.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsufs.2023.1265109/full#supplementary-material

Footnotes

1. ^We define “crop improvement priorities” as research priorities in crop improvement. These include research priorities from the domain of breeding, processing, extension and dissemination, cross-cutting themes and climate.

2. ^Here we use product concept as defined by Rutsaert et al. (2022). Product concepts are brief narrative descriptions, easy to interpret and present. They describe the morphological characteristics of the seed and plant variety, the main grower requirement that the variety addresses and conclude with an additional list of standardized information (e.g., yield potential, fertilizer needs, maturity, grain usage).

3. ^We define gender dynamics as the relationships and interactions between and among girls, boys, women, and men. These are informed by sociocultural ideas about gender and the power relationships that define them. Depending upon how they are manifested, gender dynamics can reinforce or challenge existing norms.

4. ^This is analogous to research on traits. If a study presets the list of traits that is asking about, and this is based on traits that they can currently breed for, any priority information is immediately actionable. The utilization of open ended questions would deliver an array of information more detailed on trait preferences, but qualitative data are harder to act on immediately. This is a tension to be aware of.

5. ^We advise that the list is as complete as possible. However, as the number of combinations in the experiment grow exponentially with the objects to be evaluated (and there are computational limits to consider, to keep the framework as agile as possible), we highly recommend NARs/CGIAR centers having a maximum of 12 research priorities (3 top-ranked priorities for 4 different thematic domains, excluding crop improvement).

6. ^Examples of research priorities in the domain of crop improvement are “breeding for high yield,” “breeding for biotic stresses,” “improving soil fertility,” etc.

References

Alston, J. M., Pardey, P. G., and Rao, X. (2022). Payoffs to a half century of CGIAR research. Am. J. Agric. Econ. 104, 502–529. doi: 10.1111/ajae.12255

CrossRef Full Text | Google Scholar

Avila, A. F. D., Quirino, T. R., Contini, E., and Rech Filho, E. L. (2002). “Social and economic impact ex ante evaluation of Embrapa's biotechnology research products” in Economic and social issues in agricultural biotechnology. eds. R. E. Evenson, V. Santaniello, and D. Zilberman (Wallingford, UK: CABI Publishing), 287–307.

Google Scholar

Blundo‐Canto, G., Devaux‐Spatarakis, A., Mathé, S., Faure, G., and Cerdan, G. (2020). Using a participatory theory driven evaluation approach to identify causal mechanisms in innovation processes. New Directions for Evaluation 2020, 59–72.

Google Scholar

Braunschweig, T. (2000). Priority setting in agricultural biotechnology research: supporting public decisions in developing countries with the analytic hierarchy process. Research report no. 16. The Hague: International Service for National Agricultural Research.

Google Scholar

Brown, D., Van den Bergh, I., De Bruin, S., Machida, L., and Van Etten, J. (2020). Data synthesis for crop variety evaluation. A review. Agron. Sustain. Dev. 40:25. doi: 10.1007/s13593-020-00630-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Dahlberg, K. A. (1988). Ethical and value issues in international agricultural research. Agric. Hum. Values 5, 101–111. doi: 10.1007/BF02217181

CrossRef Full Text | Google Scholar

Davis, J. S., Oram, P. A., and Ryan, J. G. (1987). Assessment of agricultural research priorities: an international perspective. ACIAR monograph No. 4. Canberra, Australia: Australian Centre for International Agricultural Research (ACIAR) and international food policy research institute (IFPRI).

Google Scholar

Doss, C., and Kieran, C. (2014). Standards for collecting sex-disaggregated data for gender analysis; a guide for CGIAR researchers. Available at: https://www.pim.cgiar.org/files/2012/05/Standards-for-Collecting-Sex-Disaggregated-Data-for-Gender-Analysis.pdf (Accessed September 11, 2023).

Google Scholar

Endresen, D. T. F., Street, K., Mackay, M., Bari, A., and De Pauw, E. (2011). Predictive association between biotic stress traits and eco-geographic data for wheat and barley landraces. Crop Sci. 51, 2036–2055. doi: 10.2135/cropsci2010.12.0717

CrossRef Full Text | Google Scholar

Fisher, M., and Carr, E. R. (2015). The influence of gendered roles and responsibilities on the adoption of technologies that mitigate drought risk: the case of drought-tolerant maize seed in eastern Uganda. Glob. Environ. Chang. 35, 82–92. doi: 10.1016/j.gloenvcha.2015.08.009

CrossRef Full Text | Google Scholar

Groot, J. C., Oomen, G. J., and Rossing, W. A. (2012). Multi-objective optimization and design of farming systems. Agric. Syst. 110, 63–77. doi: 10.1016/j.agsy.2012.03.012

CrossRef Full Text | Google Scholar

HarvestChoice. (1995). Dynamic research evaluation for management (DREAM). Harvard Dataverse.

Google Scholar

Linstone, H. A., and Turoff, M. (1975). The Delphi method. Reading, MA: Addison-Wesley.

Google Scholar

Mills, B. F. (1997). Ex-ante agricultural research evaluation with site specific technology generation: the case of sorghum in Kenya. Agric. Econ. 16, 125–138. doi: 10.1016/S0169-5150(96)01218-2

CrossRef Full Text | Google Scholar

Mills, B. F., and Karanja, D. D. (1997). Processes and methods for research programme priority setting: the experience of the Kenya Agricultural Research Institute wheat Programme. Food Policy 22, 63–79. doi: 10.1016/S0306-9192(96)00031-0

CrossRef Full Text | Google Scholar

Orr, A., Cox, C. M., Ru, Y., and Ashby, J. A. (2018). Gender and social targeting in plant breeding. Working Paper.

Google Scholar

Pemsl, D. E., Staver, C., Hareau, G., Alene, A. D., Abdoulaye, T., Kleinwechter, U., et al. (2022). Prioritizing international agricultural research investments: lessons from a global multi-crop assessment. Res. Policy 51:104473. doi: 10.1016/j.respol.2022.104473

CrossRef Full Text | Google Scholar

Petsakos, A., Hareau, G., Kleinwechter, U., Wiebe, K., and Sulser, T. B. (2018). Comparing modeling approaches for assessing priorities in international agricultural research. Res. Eval. 27, 145–156. doi: 10.1093/reseval/rvx044

PubMed Abstract | CrossRef Full Text | Google Scholar

Polar, V., Ashby, J. A., Thiele, G., and Tufan, H. (2021). When is choice empowering? Examining gender differences in varietal adoption through case studies from sub-Saharan Africa. Sustainability 13:3678. doi: 10.3390/su13073678

CrossRef Full Text | Google Scholar

Ragot, M., Bonierbale, M., and Weltzien, E. (2018). From market demand to breeding decisions: a framework. CGIAR gender and breeding initiative working paper 2. Lima, Peru: CGIAR Gender and Breeding Initiative.

Google Scholar

Raitzer, D. A., and Kelley, T. G. (2008). Benefit–cost meta-analysis of investment in the international agricultural research centers of the CGIAR. Agric. Syst. 96, 108–123. doi: 10.1016/j.agsy.2007.06.004

CrossRef Full Text | Google Scholar

Randolph, T. F., Kristjanson, P. M., Omamo, S. W., Odero, A. N., Thornton, P. K., Reid, R. S., et al. (2001). A framework for priority setting in international livestock research. Res. Eval. 10, 142–160. doi: 10.3152/147154401781777024

CrossRef Full Text | Google Scholar

Rutsaert, P., Donovan, J., Mawia, H., De Sousa, K., and Van Etten, J. (2022). Future market segments for hybrid maize in East Africa. Market intelligence brief series 2. Montpellier: CGIAR Available at: https://hdl.handle.net/10883/22467.

Google Scholar

Singh, S., et al. (2020). Krishi Vigyan Kendra knowledge network (KVK). Available at: https://kvk.icar.gov.in/aboutkvk.aspx.

Google Scholar

Steinke, J., and Van Etten, J. (2017). Gamification of farmer-participatory priority setting in plant breeding: design and validation of “AgroDuos”. J. Crop Improv. 31, 356–378. doi: 10.1080/15427528.2017.1303801

CrossRef Full Text | Google Scholar

Tarjem, I. A., Westengen, O. T., Wisborg, P., and Glaab, K. (2022). “Whose demand?” the co-construction of markets, demand and gender in development-oriented crop breeding. Agric. Hum. Values 2022, 1–18. doi: 10.1007/s10460-022-10337-y

CrossRef Full Text | Google Scholar

Thornton, P. K., Whitbread, A., Baedeker, T., Cairns, J., Claessens, L., Baethgen, W., et al. (2018). A framework for priority-setting in climate smart agriculture research. Agric. Syst. 167, 161–175. doi: 10.1016/j.agsy.2018.09.009

CrossRef Full Text | Google Scholar

Weltzien, E., Rattunde, F., Christinck, A., Isaacs, K., and Ashby, J. (2019). Gender and farmer preferences for varietal traits: evidence and issues for crop improvement. Plant Breed. Rev. 43, 243–278. doi: 10.1002/9781119616801.ch7

CrossRef Full Text | Google Scholar

Wiebe, K., Sulser, T. B., Dunston, S., Rosegrant, M. W., Fuglie, K., Willenbockel, D., et al. (2021). Modeling impacts of faster productivity growth to inform the CGIAR initiative on crops to end hunger. PLoS One 16:e0249994. doi: 10.1371/journal.pone.0249994

PubMed Abstract | CrossRef Full Text | Google Scholar

Witcombe, J. R., Joshi, K. D., Gyawali, S., Musa, A. M., Johansen, C., Virk, D. S., et al. (2005). Participatory plant breeding is better described as highly client-oriented plant breeding. I. Four indicators of client-orientation in plant breeding. Exp. Agric. 41, 299–319. doi: 10.1017/S0014479705002656

CrossRef Full Text | Google Scholar