Sublethal Endpoints in Non-target Organism Testing for Insect-Active GE Crops

Historically, genetically engineered (GE) plants that have incorporated genes conferring insect protection have primarily used Cry proteins derived from Bacillus thuringiensis (Bt) to achieve their insecticidal phenotype. As a result, regulators have developed a level of familiarity and confidence in reviewing plants incorporating these insecticidal proteins. However, new technologies have been developed that produce GE plants that incorporate pest protection by triggering an RNA interference (RNAi) response or proteins other than Bt Cry proteins. These technologies have new modes of action. Although the overall assessment paradigm for GE plants is robust, there are ongoing discussions about the appropriate tests and measurement endpoints needed to inform non-target arthropod assessment for technologies that have a different mode of action than the Bt Cry proteins. As a result, increasing attention is being paid to the use of sublethal endpoints and their value for environmental risk assessment (ERA). This review focuses on the current status and history of sublethal endpoint use in insect-active GE crops, and evaluates the future use of sublethal endpoints for new and emerging technologies. It builds upon presentations made at the Workshop on Sublethal Endpoints for Non-target Organism Testing for Non-Bt GE Crops (Washington DC, USA, 4–5 March 2019), and the discussions of government, academic and industry scientists convened for the purpose of reviewing the progress and status of sublethal endpoint testing in non-target organisms.

Introduction

The introduction of insect-resistant GE crops began in the 1990s, with a number of today's crops incorporating Bt Cry proteins (Koch et al., 2015; Naranjo et al., 2020). According to the International Service for the Acquisition of Agri-biotech Applications (ISAAA) GM Approval database¹, 85 transformation events involving Bt Cry protein expressed in 10 crops received regulatory approval somewhere in the world by the end of 2019. These have been further incorporated into 206 stacks (that combine two or more GE traits) that have received additional regulatory approvals¹. These approvals have each been accompanied by an ERA that has typically focused on identifying the target range and specificity of the Bt Cry proteins using a tiered approach to non-target testing that is very similar to the approach used in the assessment of chemical pesticides (Garcia-Alonso et al., 2006; Romeis et al., 2008; Figure 1). Tier-1 testing involves the use of surrogate species tested under worst-case exposure conditions in the laboratory to identify potential hazards to them. Specific endpoints, typically including mortality, are measured in tier-1 tests under conditions of exposure to concentrations, usually several fold higher than concentrations expected in the field (Romeis et al., 2008). In the absence of relevant negative effects in test species at high exposures, a conclusion that the likelihood of adverse ecological effects under realistic conditions is low or negligible can be supported (Romeis et al., 2013b). If negative effects are observed under worst-case conditions, then higher tier studies are conducted to establish if the effect is relevant under more realistic conditions (i.e., lower dose; Rose, 2007; EFSA, 2010; Figure 1).

FIGURE 1

Figure 1. This figure details the tiered structure of testing used during the ERA process. This is hypothesis-based testing, where the initial tests are conducted at doses higher than what is encountered in a natural setting to simulate a worst-case scenario exposure. Example risk hypotheses are provided for NTOs to demonstrate how this structure can be used to test sub-lethal effects. This figure was adapted from Romeis et al. (2008).

After more than 20 years of use in the field, there is a substantial history associated with GE plants incorporating Bt Cry proteins and their safety in the environment (Mendelsohn et al., 2003; Naranjo, 2009; Duan et al., 2010; Center for Environmental Risk Assessment, 2012; Guo et al., 2014; Koch et al., 2016; Romeis et al., 2019). The utility of a tiered approach using mortality as the primary endpoint is supported by experience with Bt Cry proteins and by an understanding of the mode of action, target specificity and exposure levels for these proteins. However, new pest control technologies, including non-Bt Cry proteins and the use of RNAi, have led to an increased interest in sublethal endpoints testing. This interest is due to several factors, including broader interest in sublethal impacts of chemicals, differences in the mode of action, and the length of time required to observe an effect, and concerns about cumulative or additive effects of multiple stressors in the environment.

While the use of sublethal endpoints is potentially informative for the ERA of non-Bt GE plants, there are a number of challenges to implementing this approach. These include the wide variety of potential endpoints from which to choose and difficulties interpreting the relationship between sublethal endpoints in laboratory studies and observable effects in the field. This paper addresses some of these challenges by examining sublethal endpoints in the context of insect-resistant GE plants and considering when inclusion of sublethal endpoints may or may not be warranted in the context of problem formulation for a case-specific ERA. This review has been informed by discussions at a workshop convened by the ILSI Research Foundation (Washington, DC, USA, 4–5 March 2019) that asked participants to consider the relevance of sublethal endpoint testing using a case study approach.

Non-Target Organism Assessment

Problem Formulation

The process of identifying and refining the information that will be informative for case-specific ERA is referred to as problem formulation (USEPA, 2016). The mechanics of problem formulation can be described in a number of ways (Hill, 2005; OGTR, 2009; Wolt et al., 2010; Gray, 2012), but the process involves a series of steps that incorporate context for the decision being made, information about the receiving environment and the societal values or protection goals that are identified in relevant laws and regulations. Because these protection goals are often broad, case specific ERA requires the refinement of operational protection goals and the subsequent identification of assessment endpoints, which allow the testing of relevant hypotheses to inform the assessment (Sanvido et al., 2012; Devos et al., 2015). Data are then collected under laboratory, semi-field or field conditions for measurement endpoints that are related to those assessment endpoints (Garcia-Alonso and Raybould, 2014). The advantage of problem formulation is that it provides an explicit rationale for how and why a particular measurement endpoint will be informative to an ERA (Figure 2).

FIGURE 2

Figure 2. This figure provides details of the broad steps used in the risk assessment process. All risk assessments start with problem formulation to develop the hypotheses that need to be tested. Once problem formulation is completed risk characterization is conducted with analysis of exposure and testing to determine the potential effects of the exposure. Data from the risk characterization studies are used in the risk evaluation process. This figure was adapted from Wolt et al. (2010).

For most risk assessments on non-target invertebrates, operational protection goals are reliant on maintaining populations of value or beneficial arthropods that contribute to important ecosystem services (Devos et al., 2015). Among the most relevant have been populations of pollinators, parasitoids, and predators as well as charismatic, protected, threatened, or endangered species, for which an exposure assessment indicates they will have a meaningful exposure to the GE plant. Once the particular species of interest are identified, appropriate surrogate, indicator, or focal species are selected for testing (Rose, 2007; EFSA, 2010).

NTO Study Design

As with any research or regulatory study, the design of an non-target organism (NTO) study must be appropriate for the intended end use of the data. Well-designed early-tier studies are intended to identify any hazards that require further study. Careful consideration, therefore, must be given to ensuring that tests are reliable and especially that false negative results (i.e., failing to identify a hazard) are avoided since they would lead to the release of hazardous material. False positive results, on the other hand, also should be avoided since they might have consequences, even beyond the triggering of additional studies (Romeis et al., 2013a). The first step is to identify appropriate surrogate species (Carstens et al., 2014). These should be chosen based on the representativeness of species of importance in the expected receiving environment, and ideally represent a functional group of interest. As a practical consideration, it is useful to select a surrogate from readily available laboratory reared strains. This provides uniformity to the test and control groups as well as reproducibility of results obtained from multiple laboratories. A surrogate species may additionally be selected based on its sensitivity to the test substance, taking into account what is known about the spectrum of activity of the insecticidal protein. For example, when the surrogate species and the target pests are more closely related phylogenetically, it is more probable that the surrogate species will be sensitive to the test substance (Romeis et al., 2013a). NTO tests should be designed to expose the surrogate to the test substance in a way that maximizes the likelihood of detecting an effect and represents a relevant exposure pathway (e.g., dietary exposure). While juveniles are usually expected to be more susceptible in some species, other species may instead or also have susceptible adults, so it is advantageous to test multiple life stages provided there is a reasonable exposure pathway for those life stages (Romeis et al., 2011, 2013b).

When designing the study protocol, the most expedient approach is to use an already available artificial diet into which both test substances and positive controls can be easily incorporated. This allows the concentrations of the test substance to exceed the expected environmental exposure in order to test levels that exceed worst-case exposures. If an artificial diet is not available, or not suitable for incorporation of the test substance, then GE plant material can be used. However, this may limit the test dose possibilities and reduces the margin of exposure. Furthermore, it might be a challenge to identify the appropriate non-transformed control material.

The use of appropriate controls is a critical component for ensuring that test results are reliable and meaningful (Romeis et al., 2011). When plant material is used as a test substance, near-isogenic lines are preferred as a control in order to eliminate confounding variables in the composition of the materials, unrelated to the GE trait. For NTO studies, negative controls should be designed to mimic the test conditions as closely as possible, and usually use the same test diet with an inert ingredient in place of the test substance. Negative controls also are essential to evaluate if specific a priori assay performance criteria were met (e.g., level of acceptable control mortality), to ensure that any observed effects are not due to nutritional inadequacies, failure of the test arthropods to eat the diet or inappropriate conditions in the experimental design for the health of the test arthropods. Similarly, positive controls are recommended to ensure that the test system is functioning as intended. This includes verifying that the test animal is consuming the test substance. To ensure this, having a positive control that closely mimics the mode of action, or at least requires the same route of exposure as the test substance is desirable. Moreover, recombinant insecticidal proteins produced in microorganisms are often used as the test substance in place of plant derived proteins because of the impracticality of purifying a sufficient quantity of protein from the plants. For this reason, it is important to assess the functional and biochemical equivalence of the test substance with the protein expressed by the GE crop (Raybould et al., 2013). Another aspect to consider is the biological activity of the test substance in the diet: a parallel bioassay is usually performed using a target species in order to check that the insecticidal protein is stable and has the expected level of biological activity in the diet. In practice, not every test system is able to meet the definition of an ideal NTO study. However, as long as the methods and protocols are reported accurately and the limitations of the study are understood and explained, these studies should still be considered in a weight-of-evidence approach. As a minimum standard, tier 1 studies should include a worst-case-exposure scenario, confirmation that the test species is exposed to the biologically active test substance, and the use of a negative experimental control (De Schrijver et al., 2016).

Measurement Endpoints in NTO Testing

Historically, the primary and most common measurement endpoint for an early tier laboratory study is mortality. There are a number of reasons for this, including that it is normally unambiguous, easy to measure and has a clear and direct relationship to potential adverse effects on populations of NTOs and ecosystem services they provide. Because it is a common endpoint, there are study designs and methods described for measuring mortality in multiple test systems and for many test species that are validated. Adherence to these standards is often encouraged or required for submitting study results associated with regulatory reviews.

One advantage of using mortality as a study endpoint is that regulatory agencies have developed policies and practices associated with interpreting the results of mortality in NTO studies in regulatory risk assessments. A white paper generated by a panel of experts suggested 50% mortality or a 50% impact on development or weight at the maximum hazard dose in tier 1 studies with insecticidal proteins as a reasonable threshold value for determining if higher tier studies will be informative (Rose, 2007). The European Food Safety Authority (EFSA) recommends a multiplicative effect of 20% in tier 1 laboratory studies in order to trigger additional studies (EFSA, 2010). One criticism of looking at mortality as a measurement endpoint is that it may not be protective for sublethal effects that might impact populations and the ecological services provided by NTOs. While this is certainly true, it is mitigated by achieving a sufficient margin of exposure, margin of safety or other conservative features of NTO study design.

While it is often remarked that mortality is the only endpoint used in in the regulatory risk assessment of Bt Cry proteins (Andow and Hilbeck, 2004; Desneux and Bernal, 2010), this is not the case. As shown in Table 1, although mortality is the most common measurement endpoint, it is not unusual to see one or more sublethal endpoints recorded for a regulatory study (e.g., larval weight and development time can be collected when conducting lethality testing provided the test is of sufficient duration for the test species to reach developmental milestones). Most regulatory studies on insecticidal proteins have also recorded sublethal endpoints, but these data are not always reported in summaries or subsequent representation of the study. Sublethal endpoints that have been measured in studies for Bt Cry proteins include larval and adult weight, developmental timing, fecundity (number of offspring), percent completing adult development and even mobility. What is equally apparent from Table 1 is that while sublethal endpoints have been measured, sometimes it is not immediately obvious how or why particular data on sublethal endpoints are collected, and there is little consistency in how those endpoints are reported in the literature. This is not unexpected as these measurements are selected and coordinated by individual product managers in the absence of specific data requirements. Standardized and validated test protocols used to assess foliar applications of pesticides published by the International Organization for Biological and Integrated Control of Noxious Animals and Weeds (IOBC) include methods for a total of nine beneficial species (two parasitoids, seven predators) and can inform NTO testing for GE plants. With the exception of one beetle species (Aleochara bilineata, Coleoptera: Staphylinidae), all IOBC protocols assess mortality as an endpoint (Candolfi et al., 2000). In addition, the tests also considered sublethal endpoints, mainly based on reproduction but also food consumption and behavior of the organism.

TABLE 1

Table 1. Laboratory studies with beneficial non-target invertebrates (predators, parasitoids, pollinators) or surrogate species for the soil and aquatic environment to support the regulatory risk assessments of plant expressed insecticidal Bt Cry proteins.

The selection of measurement endpoints associated with NTO assessment should be guided by a proper problem formulation that incorporates case-specific information pertinent to the assessment. In this way, measurement endpoints are selected that are clearly linked to an assessment endpoint and associated with the risk hypothesis. Risk hypotheses should not be generic, but rather case-specific and informed by what is known about the insect active routes of exposure and its effects on sensitive or target species. Also, prior to the study, the project managers and risk assessors should know specifically how the measurement endpoints will be interpreted and how they will be used in the decision process. If it cannot be clearly articulated how study results will inform the assessment, then the measurement endpoints for the study may not be well-aligned with protection goals and decision-making priorities. Finally, the measurement endpoints should be designed to incorporate the practical realities and limitations associated with available and appropriate relevant test species.

Experience With Sublethal Endpoint Testing in Non-Bt GE Crops

The development of novel, non-Bt Cry proteins for insect control and of non-protein based methods such as RNAi has been accompanied by an increased interest in assessing sublethal impacts. Publications related to these new technologies demonstrate some of the ways that sublethal endpoints are used.

RNAi and MON87411

The potential use of RNAi for pest control has been widely discussed (Burand and Hunter, 2013) and regulators are considering the use of RNAi to control insect pest in relation to risk assessment (USEPA, 2014; Casacuberta et al., 2015; Roberts et al., 2015). Presently, a single insect protected crop using RNAi as a mechanism has been approved for commercialization. The effect of the RNAi in the target pest, the western corn rootworm (WCR, Diabrotica virgifera virgifera), is triggered by the presence of a dsRNA targeting a housekeeping gene. MON87411 targets the WCR Snf7 gene and its mode of action has been well-characterized (Bolognesi et al., 2012; Ramaseshadri et al., 2013). Uptake of DvSnf7 RNA generates eventual mortality or severe growth inhibition, which can be observed 5 days after exposure to the test substance. There is also a relationship between exposure duration and eventual growth inhibition at lower doses, but short exposures at high doses are sufficient to induce mortality (Bolognesi et al., 2012). A description of the published ERA for MON87411 expressing DvSnf7 RNA, includes a description of NTO testing conducted in support of the risk assessment (USEPA, 2015; Bachman et al., 2016). A range of vertebrate tests were conducted, as well as arthropod testing including toxicity assays for seven species of arthropod. In addition to mortality, sublethal endpoints were observed for each species and included measures of time to adulthood, percent adult emergence, adult biomass (weight), and fecundity (number of surviving offspring produced). Tests were conducted with concentrations of the test substance in excess of 10-fold the maximum expected environmental concentration and the duration of the test period was in excess of the time required to see an effect in the target species. The selection of surrogate species took into account the mode of action, considering that coleopterans (beetles) show significantly greater sensitivity to ingested dsRNA than other arthropod orders (Roberts et al., 2015). Because this order of insect includes the target species, WCR, it provides a good illustration of how phylogenetic relationships and an understanding of the mode of action can facilitate the choice of surrogates. Additionally, bioinformatics was used as a complementary tool to perform a screening and to identify potential surrogate species based upon the presence of sequence matches. Sequence alignment between the genome and DvSnf7 informed the number and type of species tested, focusing on those which were considered most likely to be informative (Bachman et al., 2016).

Non-Bt Cry Insecticidal Proteins: IPD072Aa

One example of a non-Bt insecticidal protein that has been subject to extensive NTO assessment is IPD072Aa, isolated from Pseudomonas chlororaphis, which has activity against WCR (Schellenberger et al., 2016). The IPD072Aa protein has been the subject of bioassays to determine the spectrum of activity in order to inform the NTO risk assessment (Boeckman et al., 2019). As the target pest is a coleopteran, bioassays were conducted with 11 species of Coleoptera, representing four families. Additionally, four species of lepidopteran (moths and butterflies) representing four families in this order were tested. Measurement endpoints included mortality as well as weight for all but one species tested, and time to emergence for two species of ladybird beetles. No observed effects were reported for any of the Lepidoptera species, but both mortality and sublethal effects were observed at varying protein concentrations in some of the tested Coleoptera. In line with best practices for NTO study design (see NTO Study Design above) the criteria for selection of species to characterize the spectrum of activity of IPD072Aa was based on several factors such as the phylogenetic relationship between the species and WCR, established laboratory bioassay methodologies, availability of laboratory reared insects, a known suitable diet and reproducibility of the measurement endpoints (Boeckman et al., 2019).

IPD072Aa has a midgut site of action (SOA) where it targets and disrupts midgut epithelial cells causing breakdown of the epithelial lining in WCR through what appears to be a non-pore forming mechanism (Schellenberger et al., 2016). The ability of IPD072Aa to kill WCR larvae resistant to mCry3A or Cry34Ab1/Cry35Ab1 indicates that its target site differs from those of the Bt Cry proteins (Carlson et al., 2019). This knowledge related to its mode of action and the phylogentic relationship between the candidate surrogate species and WCR will guide selection of the appropriate surrogate species and measurement endpoints in NTO assays.

Non-Cry Insecticidal Proteins: Vip3A

Vip3A is a Bt vegetative insecticidal protein that is active against lepidopteran pests. It has a different mode of action from Cry proteins, and when delivered as a combined treatment it has the potential to delay the evolution of pest resistance to Bt crops (Lee et al., 2003). A description of the ERA for MIR162, a maize event expressing the Vip3A protein, has been published (Raybould and Vlachos, 2011). The bioassays were conducted using species representing functional groups of foliar arthropods, soil-dwelling invertebrates, pollinators, wild birds and mammals, aquatic invertebrates and farmed or wild fish (Raybould and Vlachos, 2011). In addition to mortality, sublethal endpoints such as fecundity, weight increase, adult emergence, body weight, and length were recorded for some of the species tested. Depending on the species, worst-case, or conservative maximum expected environmental concentrations were used.

Discussion

Sublethal Endpoints Are Addressed in the Context of Existing Frameworks for NTO Assessment

The rationale for conducting NTO studies in support of regulatory risk assessments is not dependent on whether the studies are designed to measure mortality or sublethal endpoints. Before a study is conducted, the problem formulation process should identify a set of informative tests based on the pathways of environmental exposure, and an identification of relevant taxa and functional groups for which risk should be assessed. Once the appropriate tests are identified, consideration can be given to what sort of measurement endpoints will best inform the assessment. When making these decisions, it is important to keep in mind the practical limitation associated with NTO testing and risk assessment.

Absence of a Single Definitive Test or Sublethal Endpoint

As with other types of testing, there is no single definitive test that can address every possibility of sublethal effects. As is typical of the regulatory risk assessment process, consideration of whether a test is necessary should be done on a case-by-case basis, focusing on the value of the information for reaching conclusions about overall risk. Although flexibility in testing allows sublethal tests to be tailored to specific needs of a chosen test system, non-uniformity in testing procedures can make it difficult to compare results obtained by independent researchers conducting experiments and then reporting sublethal impacts in the literature. These comparisons may provide helpful context for regulatory considerations. The endpoints that have been previously utilized to measure sublethal effects for all types of insect-active GE plants (i.e., weight, growth and developmental time, fecundity) appear to be useful and sufficient for sublethal testing of non-Bt Cry insect actives, both protein-based and dsRNA-based.

Effect of the Active on the Target Can Indicate the Utility of Sublethal Endpoints in NTO Testing

Knowledge regarding the mode of action and time to effect, as it relates to the effect of the insect active on the target insect, is instructive for ascertaining whether sublethal endpoint testing is likely warranted in NTOs. For example, if an insect active targets a cellular process that broadly affects growth (e.g., protein synthesis) it may be reasonably expected that collection of data measuring sublethal endpoints such as weight, development time, and reproduction may be warranted. If those endpoints are affected in the target organism, they may also be affected in NTOs. Additionally, the time that it takes for sublethal effects to manifest in the target organism during lethality testing may also be an indication that sublethal endpoint testing is warranted for NTO assessment. If mortality in the target organism is delayed, but indications of mortality are evident earlier due to the onset of sublethal effects such as delayed growth or development, then it may make sense to look for these effects in NTO assessment as well.

Sublethal Endpoints Need to Relate to Ecologically Relevant Assessment Endpoints

Sublethal endpoints that relate to measurable ecologically relevant endpoints will be much easier to interpret than more complicated endpoints. Typically, measurement endpoints for sublethal effects that include development time, growth/weight and reproduction are used for data collection, and these endpoints are readily quantifiable and relatable to assessment endpoints such as population size. While a variety of additional sublethal endpoints reflecting more ambiguous measurements have been reported in the published literature (i.e., feeding behavior and learning performance; Ramirez-Romero et al., 2008), the interpretation of these endpoints with respect to ecological outcomes is challenging.

Practical Considerations for Developing New Test Systems

Any NTO testing is dependent on the existence or the development of a well-characterized and validated test system. Some existing test systems may lend themselves to the collection of sublethal endpoint data, but others may not. Thus, development of new test systems may be required when and if the problem formulation indicates that a particular insect active is likely to cause sublethal effects in relevant NTOs. In either case, before the tests are conducted it is important to assure that the results of the sublethal measurements will be meaningful for risk assessment.

Addressing Knowledge Deficits

For RNAi-based insect actives, bioinformatics will become increasingly important in predicting potential adverse effects associated with exposure to dsRNA. However, there are currently significant gaps in bioinformatics data for both pest organisms and for ecologically important taxonomic groups, as well as a lack of information on which taxonomic groups are (in)sensitive to environmental RNAi. Narrowing those information gaps will provide a better understanding of the extent to which different NTOs need to be assessed for a case-specific assessment for a particular environmental RNAi.

More sublethal endpoint data are collected than is widely acknowledged. In part, this is, due to the primary status of mortality in testing guidelines and in summary reports or journal publications where sublethal endpoint measurements often are relegated to supplementary material. Finding new means of sharing this information (and improving access to it) is needed in the future to increase the potential usefulness of this information for risk assessments. Similarly, data collected during early characterization of pesticidal proteins may be considered proprietary in nature, presenting a barrier to broad distribution. Whenever possible, however, mechanisms should be encouraged to improve access to this type of information. There are several recent examples where early characterization of pesticidal molecules have been published (Bachman et al., 2013, 2017; Boeckman et al., 2019). However, the research community is encouraged to find new means of storing and disseminating information that often is omitted from peer-reviewed publications but has value as a collective resource in further development of NTO risk assessment methodology.

Conclusions

NTO testing is conducted in support of regulatory decision making, and therefore must be designed for this purpose, rather than simply to conduct scientifically interesting experiments (Raybould, 2010). The testing associated with any particular insect-active GE crop should be informed by a problem formulation process. Problem formulation takes into account what is known about the insect-active protein, the crop, and the expected interactions between NTOs and the associated insect-active crop, as well as the availability of well-developed test systems that facilitate the interpretation of test results in a regulatory context. The problem formulation process remains fundamentally the same whether the measurement endpoint is mortality or sublethal endpoints. When selecting sublethal endpoints for consideration, a risk hypothesis should link the sublethal endpoint to an assessment endpoint and the associated protection goal. Because most NTOs are protected at the population level and NTO communities at the functional level, typically, sublethal endpoints that are related to growth, development, and reproduction and which can be easily extrapolated to population level effects are most informative. While additional sublethal endpoints might be measurable, they should only be considered for regulatory testing if there is a clear relationship to a protection goal and the results are likely to reduce uncertainties associated with the NTO assessment. A review of recent and past measurement of sublethal endpoints collected to inform regulatory studies of plant incorporated insecticidal Bt Cry proteins is summarized in Table 1. These data suggest sublethal endpoints for current insect resistant GE crops are observed and measured more routinely than is often claimed in the literature.

A long history of standardization exists that can inform the future of NTO testing. While standard methods are not absolutely required for testing possible sublethal impacts, such studies can be informative for risk assessment. However, for these studies to be informative, there should be a clear understanding of what data are being collected and what is the rationale for collecting them. When published in peer reviewed publications, these sublethal endpoints are often published as supplemental data. If sublethal testing is done, and data are reported, these data should be presented more prominently in research reports. This practice would promote a broader understanding and further discussion of the utility of sublethal endpoints and enhance their usefulness to the risk assessment process.

Disclosure

The information and views are those of the authors as individuals and experts in the field, and do not necessarily represent those of the organizations where they work. Mention of a proprietary product does not constitute an endorsement or a recommendation for its use by Greenlight Biosciences, Inc., Iowa State University, ILSI Research Foundation, Newcastle University, The University of Arizona, University of Kentucky, University of Nebraska-Lincoln, or USDA.

Author Contributions

JR, CB, RH, MM, FV, and AR lead the initial drafting of the manuscript. CB, JB, ME, RH, SL, MM, RM, JT, TR, AV, JR, FV, XZ, and AR all made significant contributions to the drafting of the article and developing the table. CB, JB, ME, RH, SL, MM, TR, AV, JR, FV, XZ, and AR attended the ABSTC sponsored workshop. JT organized the ABSTC sponsored workshop. RM, AV, and AR developed and created the figures. RM and AR lead the editorial efforts to develop the manuscript.

Funding

This work and workshop were funded by a grant from the Agricultural Biotechnology Stewardship Technical Committee (ABSTC) NTO Committee.

Conflict of Interest

CB is employed by Corteva Agriscience, SL is employed by Bayer Crop Science, and TR is employed by Greenlight Biosciences, Inc. AV has received funding from Bayer Crop Science and Corteva Agriscience to perform research on RNAi for insect pest management and has six issued RNAi patents.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^http://www.isaaa.org/gmapprovaldatabase/

References

Andow, D. A., and Hilbeck, A. (2004). Science-based risk assessment for nontarget effects of transgenic crops. Bioscience 54, 637–649. doi: 10.1641/0006-3568(2004)054[0637:SRAFNE]2.0.CO;2

CrossRef Full Text | Google Scholar

Bachman, P. M., Ahmad, A., Ahrens, J. E., Akbar, W., Baum, J. A., Brown, S., et al. (2017). Characterization of the activity spectrum of MON 88702 and the plant-incorporated protectant Cry51Aa2.834_16. PLoS ONE 12:e0169409. doi: 10.1371/journal.pone.0169409

PubMed Abstract | CrossRef Full Text | Google Scholar

Bachman, P. M., Bolognesi, R., Moar, W. J., Mueller, G. M., Paradise, M. S., Ramaseshadri, P., et al. (2013). Characterization of the spectrum of insecticidal activity of a double-stranded RNA with targeted activity against Western corn rootworm (Diabrotica virgifera virgifera LeConte). Transgenic Res. 22, 1207–1222. doi: 10.1007/s11248-013-9716-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Bachman, P. M., Huizinga, K. M., Jensen, P. D., Mueller, G., Tan, J., Uffman, J. P., et al. (2016). Ecological risk assessment for DvSnf7 RNA: A plant-incorporated protectant with targeted activity against western corn rootworm. Regul. Toxicol. Pharmacol. 81, 77–88. doi: 10.1016/j.yrtph.2016.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

Boeckman, C. J., Huang, E., Sturtz, K., Walker, C., Woods, R., and Zhang, J. (2019). Characterization of the spectrum of insecticidal activity for IPD072Aa: A protein derived from Pseudomonas chlororaphis with activity against Diabrotica virgifera virgifera (Coleoptera: Chrysomelidae). J. Econ. Entomol. 112, 1190–1196. doi: 10.1093/jee/toz029

PubMed Abstract | CrossRef Full Text | Google Scholar

Bolognesi, R., Ramaseshadri, P., Anderson, J., Bachman, P., Clinton, W., Flannagan, R., et al. (2012). Characterizing the mechanism of action of double-stranded RNA activity against western corn rootworm (Diabrotica virgifera virgifera LeConte). PLoS ONE 7:e47534. doi: 10.1371/journal.pone.0047534

PubMed Abstract | CrossRef Full Text | Google Scholar

Burand, J. P., and Hunter, W. B. (2013). RNAi: Future in insect management. J. Invertebr. Pathol. 112, S68–S74. doi: 10.1016/j.jip.2012.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar

Candolfi, M., Blümel, S., Forster, R., Bakker, F., Grimm, C., Hassan, S., et al. (2000). Guidelines to Evaluate Side-effects of Plant Protection Products to Non-target Arthropods. International Organization for Biological and Integrated Control of Noxious Animals and Weeds. Gent. Available online at: https://www.iobc-wprs.org/pub/book2000.pdf (accessed March 6, 2020).

Google Scholar

Carlson, A. B., Mathesius, C. A., Ballou, S., Boeckman, C. J., Gunderson, T. A., Mirsky, H. P., et al. (2019). Safety assessment of coleopteran active IPD072Aa protein from Pseudomonas chlororaphis. Food Chem. Toxicol. 129, 376–381. doi: 10.1016/j.fct.2019.04.055

PubMed Abstract | CrossRef Full Text | Google Scholar

Carstens, K., Cayabyab, B., De Schrijver, A., Gadaleta, P. G., Hellmich, R. L., Romeis, J., et al. (2014). Surrogate species selection for assessing potential adverse environmental impacts of genetically engineered insect-resistant plants on non-target organisms. GM Crops Food 5, 11–15. doi: 10.4161/gmcr.26560

PubMed Abstract | CrossRef Full Text | Google Scholar

Casacuberta, J. M., Devos, Y., du Jardin, P., Ramon, M., Vaucheret, H., and Nogué, F. (2015). Biotechnological uses of RNAi in plants: Risk assessment considerations. Trends Biotechnol. 33, 145–147. doi: 10.1016/j.tibtech.2014.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Center for Environmental Risk Assessment (2012). A review of the environmental safety of the Cry1Ab protein. Environ. Biosafety Res. 10, 51–71. doi: 10.1051/ebr/2012003

CrossRef Full Text | Google Scholar

De Schrijver, A., Devos, Y., De Clercq, P., Gathmann, A., and Romeis, J. (2016). Quality of laboratory studies assessing effects of Bt-proteins on non-target organisms: minimal criteria for acceptability. Transgenic Res. 25, 395–411. doi: 10.1007/s11248-016-9950-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Desneux, N., and Bernal, J. S. (2010). Genetically modified crops deserve greater ecotoxicological scrutiny. Ecotoxicology 19, 1642–1644. doi: 10.1007/s10646-010-0550-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Devos, Y., de Schrijver, A., de Clercq, P., Kiss, J., and Romeis, J. (2012). Bt-maize event MON 88017 expressing Cry3Bb1 does not cause harm to non-target organisms. Transgenic Res. 21, 1191–1214. doi: 10.1007/s11248-012-9617-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Devos, Y., Romeis, J., Luttik, R., Maggiore, A., Perry, J. N., Schoonjans, R., et al. (2015). Optimising environmental risk assessments. EMBO Rep. 16, 1060–1063. doi: 10.15252/embr.201540874

PubMed Abstract | CrossRef Full Text | Google Scholar

Duan, J. J., Head, G., McKee, M. J., Nickson, T. E., Martin, J. W., and Sayegh, F. S. (2002). Evaluation of dietary effects of transgenic corn pollen expressing Cry3Bb1 protein on a non-target ladybird beetle, Coleomegilla maculata. Entomol. Exp. Appl. 104, 271–280. doi: 10.1046/j.1570-7458.2002.01013.x

CrossRef Full Text | Google Scholar

Duan, J. J., Lundgren, J. G., Naranjo, S., and Marvier, M. (2010). Extrapolating non-target risk of Bt crops from laboratory to field. Biol. Lett. 6, 74–77. doi: 10.1098/rsbl.2009.0612

PubMed Abstract | CrossRef Full Text | Google Scholar

Duan, J. J., Paradise, M. S., Lundgren, J. G., Bookout, J. T., Jiang, C., and Wiedenmann, R. N. (2006). Assessing nontarget impacts of Bt corn resistant to corn rootworms: Tier-1 testing with larvae of Poecilus chalcites (Coleoptera: Carabidae). Environ. Entomol. 35, 135–142. doi: 10.1603/0046-225X-35.1.135

PubMed Abstract | CrossRef Full Text | Google Scholar

Duan, J. J., Teixeira, D., Huesing, J. E., and Jiang, C. (2008). Assessing the risk to nontarget organisms from Bt corn resistant to corn rootworms (Coleoptera: Chrysomelidae): Tier-I testing with Orius insidiosus (Heteroptera: Anthocoridae). Environ. Entomol. 37, 838–844. doi: 10.1603/0046-225X(2008)37[838:ATRTNO]2.0.CO;2

PubMed Abstract | CrossRef Full Text | Google Scholar

EFSA (2010). Guidance on the environmental risk assessment of genetically modified plants. EFSA J. 8:1879. doi: 10.2903/j.efsa.2010.1879

PubMed Abstract | CrossRef Full Text | Google Scholar

Garcia-Alonso, M., Jacobs, E., Raybould, A., Nickson, T. E., Sowig, P., Willekens, H., et al. (2006). A tiered system for assessing the risk of genetically modified plants to non-target organisms. Environ. Biosafety Res. 5, 57–65. doi: 10.1051/ebr:2006018

PubMed Abstract | CrossRef Full Text | Google Scholar

Garcia-Alonso, M., and Raybould, A. (2014). Protection goals in environmental risk assessment: A practical approach. Transgenic Res. 23, 945–956. doi: 10.1007/s11248-013-9760-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Gray, A. (2012). Problem formulation in environmental risk assessment for genetically modified crops: A practitioner's approach. Collect. Biosaf. Rev. 6, 1–56.

Google Scholar

Guo, Y., Feng, Y., Ge, Y., Tetreau, G., Chen, X., Dong, X., et al. (2014). The cultivation of Bt corn producing Cry1Ac toxins does not adversely affect non-target arthropods. PLoS ONE 9:e1142281. doi: 10.1371/journal.pone.0114228

PubMed Abstract | CrossRef Full Text | Google Scholar

Hill, R. A. (2005). Conceptualizing risk assessment methodology for genetically modified organisms. Environ. Biosaf. Res. 4, 67–70. doi: 10.1051/ebr:2005012

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, M. S., DeSesso, J. M., Williams, A. L., Michalek, S., and Hammond, B. (2016). Adaptation of the ToxRTool to assess the reliability of toxicology studies conducted with genetically modified crops and implications for future safety testing. Crit. Rev. Food Sci. Nutr. 56, 512–526. doi: 10.1080/10408398.2013.788994

PubMed Abstract | CrossRef Full Text | Google Scholar

Koch, M. S., Ward, J. M., Levine, S. L., Baum, J. A., Vicini, J. L., and Hammond, B. G. (2015). The food and environmental safety of Bt crops. Front. Plant Sci. 6:283. doi: 10.3389/fpls.2015.00283

PubMed Abstract | CrossRef Full Text | Google Scholar

Lee, M. K., Walters, F. S., Hart, H., Palekar, N., and Chen, J-S. (2003). The mode of action of the Bacillus thuringiensis vegetative insecticidal protein Vip3A differs from that of Cry1Ab d-endotoxin. Appl. Environ. Microbiol. 69, 4648–4657. doi: 10.1128/AEM.69.8.4648-4657.2003

PubMed Abstract | CrossRef Full Text | Google Scholar

Mendelsohn, M., Kough, J., Vaituzis, Z., and Matthews, K. (2003). Are Bt crops safe? Nat. Biotechnol. 21, 1003–1009. doi: 10.1038/nbt0903-1003

PubMed Abstract | CrossRef Full Text | Google Scholar

Naranjo, S., Hellmich, R., Romeis, J., Shelton, A., and Vélez, A. (2020). “The role and use of genetically engineered insect-resistant crops in integrated pest management systems,” in Integrated Management of Insect Pests: Current and Future Developments, eds M. Kogan and E. E. A Prof and Heinrichs (Cambridge: Burleigh Dodds Science Publishing), 283–340. doi: 10.19103/AS.2019.0047.10

CrossRef Full Text | Google Scholar

Naranjo, S. E. (2009). Impacts of Bt crops on non-target invertebrates and insecticide use patterns. CAB Rev. Perspect. Agric. Vet. Sci. Nutr. Nat. Resour. 4:11. doi: 10.1079/PAVSNNR20094011

CrossRef Full Text | Google Scholar

OGTR (2009). Risk Analysis Framework. Available online at: http://www.ogtr.gov.au/internet/ogtr/publishing.nsf/Content/risk-analysis-framework (accessed March 6, 2020).

Ramaseshadri, P., Segers, G., Flannagan, R., Wiggins, E., Clinton, W., Ilagan, O., et al. (2013). Physiological and cellular responses caused by RNAi- mediated suppression of Snf7 orthologue in western corn rootworm (Diabrotica virgifera virgifera) larvae. PLoS ONE 8:e54270. doi: 10.1371/journal.pone.0054270

PubMed Abstract | CrossRef Full Text | Google Scholar

Ramirez-Romero, R., Desneux, N., Decourtye, A., Chaffiol, A., and Pham-Delègue, M. H. (2008). Does Cry1Ab protein affect learning performances of the honey bee Apis mellifera L. (Hymenoptera, Apidae)? Ecotoxicol. Environ. Saf. 70, 327–333. doi: 10.1016/j.ecoenv.2007.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Raybould, A. (2007). Ecological versus ecotoxicological methods for assessing the environmental risks of transgenic crops. Plant Sci. 173, 589–602. doi: 10.1016/j.plantsci.2007.09.003

CrossRef Full Text | Google Scholar

Raybould, A. (2010). Reducing uncertainty in regulatory decision-making for transgenic crops: more ecological research or clearer environmental risk assessment? GM Crops 1, 25–31. doi: 10.4161/gmcr.1.1.9776

CrossRef Full Text | Google Scholar

Raybould, A., Kilby, P., and Graser, G. (2013). Characterising microbial protein test substances and establishing their equivalence with plant-produced proteins for use in risk assessments of transgenic crops. Transgenic Res. 22, 445–460. doi: 10.1007/s11248-012-9658-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Raybould, A., Stacey, D., Vlachos, D., Graser, G., Li, X., and Joseph, R. (2007). Non-target organism risk assessment of MIR604 maize expressing mCry3A for control of corn rootworm. J. Appl. Entomol. 131, 391–399. doi: 10.1111/j.1439-0418.2007.01200.x

CrossRef Full Text | Google Scholar

Raybould, A., and Vlachos, D. (2011). Non-target organism effects tests on Vip3A and their application to the ecological risk assessment for cultivation of MIR162 maize. Transgenic Res. 20, 599–611. doi: 10.1007/s11248-010-9442-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Roberts, A. F., Devos, Y., Lemgo, G. N. Y., and Zhou, X. (2015). Biosafety research for non-target organism risk assessment of RNAi-based GE plants. Front. Plant Sci. 6:958. doi: 10.3389/fpls.2015.00958

PubMed Abstract | CrossRef Full Text | Google Scholar

Romeis, J., Bartsch, D., Bigler, F., Candolfi, M. P., Gielkens, M. M. C., Hartley, S. E., et al. (2008). Assessment of risk of insect-resistant transgenic crops to nontarget arthropods. Nat. Biotechnol. 26, 203–208. doi: 10.1038/nbt1381

PubMed Abstract | CrossRef Full Text | Google Scholar

Romeis, J., Hellmich, R. L., Candolfi, M. P., Carstens, K., de Schrijver, A., Gatehouse, A. M. R., et al. (2011). Recommendations for the design of laboratory studies on non-target arthropods for risk assessment of genetically engineered plants. Transgenic Res. 20, 1–22. doi: 10.1007/s11248-010-9446-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Romeis, J., McLean, M., and Shelton, A. M. (2013a). When bad science makes good headlines: Bt maize and regulatory bans. Nat. Biotechnol. 31, 386–387. doi: 10.1038/nbt.2578

PubMed Abstract | CrossRef Full Text | Google Scholar

Romeis, J., Naranjo, S. E., Meissle, M., and Shelton, A. M. (2019). Genetically engineered crops help support conservation biological control. Biol. Control 130, 136–154. doi: 10.1016/j.biocontrol.2018.10.001

CrossRef Full Text | Google Scholar

Romeis, J., Raybould, A., Bigler, F., Candolfi, M. P., Hellmich, R. L., Huesing, J. E., et al. (2013b). Deriving criteria to select arthropod species for laboratory tests to assess the ecological risks from cultivating arthropod-resistant genetically engineered crops. Chemosphere 90, 901–909. doi: 10.1016/j.chemosphere.2012.09.035

PubMed Abstract | CrossRef Full Text | Google Scholar

Rose, R. I. (ed.). (2007). White Paper on Tier-Based Testing for the Effects of Proteinaceous Insecticidal Plant-Incorporated Protectants on Non-Target Arthropods for Regulatory Risk Assessments U.S. Environmental Protection Agency. Available online at: https://www.epa.gov/sites/production/files/2015-09/documents/tier-based-testing.pdf (accessed March 6, 2020).

Google Scholar

Sanvido, O., Romeis, J., Gathmann, A., Gielkens, M., Raybould, A., and Bigler, F. (2012). Evaluating environmental risks of genetically modified crops: Ecological harm criteria for regulatory decision-making. Environ. Sci. Policy 15, 82–91. doi: 10.1016/j.envsci.2011.08.006

CrossRef Full Text | Google Scholar

Schellenberger, U., Oral, J., Rosen, B. A., Wei, J. Z., Zhu, G., Xie, W., et al. (2016). A selective insecticidal protein from Pseudomonas for controlling corn rootworms. Science 354, 634–637. doi: 10.1126/science.aaf6056

PubMed Abstract | CrossRef Full Text | Google Scholar

Stacey, D., Graser, G., Mead-Briggs, M., and Raybould, A. (2006). Testing the impact on nontarget organisms of insecticidal proteins expressed in transgenic crops. Bull. IOBC/WPRS 29, 165–172.

Google Scholar

USEPA (2014). Meeting Materials for the January 28, 2014 Scientific Advisory Panel. Scientific Advisory Panel Minutes No. 2014-02. Available online at: https://www.epa.gov/sap/meeting-materials-january-28-2014-scientific-advisory-panel. 1–77 (accessed March 6, 2020).

USEPA (2015). Notice of Pesticide Registration. EPA Reg. No. 524e618, MON 87411. Available online at: https://www3.epa.gov/pesticides/chem_search/ppls/000524-00618-20151029.pdf (accessed March 6, 2020).

USEPA (2016). Technical Overview of Ecological Risk Assessment: Problem Formulation. Available online at: https://www.epa.gov/pesticide-science-and-assessing-pesticide-risks/technical-overview-ecological-risk-assessment (accessed March 1, 2020).

PubMed Abstract

Wolt, J. D., Keese, P., Raybould, A., Fitzpatrick, J. W., Burachik, M., Gray, A., et al. (2010). Problem formulation in the environmental risk assessment for genetically modified plants. Transgenic Res. 19, 425–436. doi: 10.1007/s11248-009-9321-9

PubMed Abstract | CrossRef Full Text | Google Scholar