Risk factors associated with the prevalence of Shiga-toxin-producing Escherichia coli in manured soils on certified organic farms in four regions of the USA

Introduction: Biological soil amendments of animal origin (BSAAO), including untreated amendments are often used to improve soil fertility and are particularly important in organic agriculture. However, application of untreated manure on cropland can potentially introduce foodborne pathogens into the soil and onto produce. Certified organic farms follow the USDA National Organic Program (NOP) standards that stipulate a 90- or 120-day interval between application of untreated manure and crop harvest, depending on whether the edible portion of the crop directly contacts the soil. This time-interval metric is based on environmental factors and does not consider a multitude of factors that might affect the survival of the main pathogens of concern. The objective of this study was to assess predictors for the prevalence of Shiga-toxin-producing Escherichia coli (non-O157 STEC) in soils amended with untreated manure on USDA-NOP certified farms.

Methods: A longitudinal, multi-regional study was conducted on 19 farms in four USA regions for two growing seasons (2017–2018). Untreated manure (cattle, horse, and poultry), soil, and irrigation water samples were collected and enrichment cultured for non-O157 STEC. Mixed effects logistic regression models were used to analyze the predictors of non-O157 STEC in the soil up to 180 days post-manure application.

Results and discussion: Results show that farm management practices (previous use with livestock, presence of animal feces on the field, season of manure application) and soil characteristics (presence of generic E. coli in the soil, soil moisture, sodium) increased the odds of STEC-positive soil samples. Manure application method and snowfall decreased the odds of detecting STEC in the soil. Time-variant predictors (year and sampling day) affected the presence of STEC. This study shows that a single metric, such as the time interval between application of untreated manure and crop harvest, may not be sufficient to reduce the food safety risks from untreated manure, and additional environmental and farm-management practices should also be considered. These findings are of particular importance because they provide multi-regional baseline data relating to current NOP wait-time standards. They can therefore contribute to the development of strategies to reduce pathogen persistence that may contribute to contamination of fresh produce typically eaten raw from NOP-certified farms using untreated manure.

1. Introduction

The consumption of organic food has grown exponentially worldwide in the past decade (Rana and Paul, 2017). The US, in particular, has seen a rapid increase in organic production, with an estimated area of 2.2 million hectares in organic production and $43.3 billion in sales (Haumann, 2017). Organic agriculture is based on sustainable and ecological principles, and the use of synthetic fertilizers and synthetic pesticides is carefully managed on organic farms. Instead, USDA-National Organic Program (NOP)—certified farms rely on biological soil amendments of animal origin (BSAAO; i.e., animal manure, compost) in combination with other ecological nutrient management techniques, including the use of locally adapted plant varieties, intercropping with nitrogen-fixing crops and trees, crop rotations, and/or cover crops (USDA-AMS, 2000; Rosen and Bierman, 2005; Rosen and Allan, 2007; USDA-NOP, 2011; Pires et al., 2018). While many crop-based agricultural operations utilize BSAAO, they are particularly important to organic farmers seeking or operating within NOP standards because the USDA-NOP prohibits the use of synthetic chemical fertilizers (USDA-AMS, 2000). Because manure from livestock species may carry foodborne pathogens, including E. coli O157:H7, Salmonella spp., Campylobacter spp., and Listeria monocytogenes, there is an increased risk of pathogen contamination of fresh produce and fruits typically consumed raw when untreated manure is applied to crop fields (Olaimat and Holley, 2012). These major pathogens have been frequently linked to outbreaks in fresh produce (Scallan et al., 2011; Luna et al., 2018; Turner et al., 2019), and traced to livestock (Jay et al., 2007). The 2018 E. coli O157:H7 romaine lettuce outbreak associated with contaminated agricultural water potentially impacted by a nearby large animal feeding operation highlights the need to investigate the co-existence of animals and other potential sources, such as BSAAO applications (FDA, 2019). The current USDA-NOP standards establish intervals between the application and incorporation of animal-based soil amendments (i.e., raw manure, untreated manure) and the time of crop harvest. At least 120 days between application (and incorporation) and harvest is required when the edible portion of the crop directly contacts the manure-amended soil, or 90 days when there is no contact (USDA-NOP, 2011; FDA, 2018). This interval between incorporation of manure and harvest was developed under production cycle criteria and not on microbial contamination risk (Natvig et al., 2002; Ingham et al., 2005). If these management practices cannot be followed, the USDA-NOP requires that raw manure be composted (USDA-NOP, 2011; FDA, 2018). The Food Safety Modernization Act (FSMA) Produce Safety Rule (PSR) does not object to the USDA-NOP standards until more data is collected, and is reserving the final decision on untreated BSAAO application intervals until more data is available (FDA, 2015, 2018). However, these application intervals do not take into account a multitude of factors that may affect the survival of foodborne pathogens, including biological and physico-chemical composition of soil and manure, livestock species, pathogen, manure (e.g., type, management), soil type (e.g., texture, moisture, organic matter, nutrients), and environmental conditions (e.g., season, ambient temperature, rainfall, humidity, sunlight) (Hutchison et al., 2005; Sinton et al., 2007; Moriarty et al., 2011; Berry et al., 2013; Harris et al., 2013; Erickson et al., 2014, 2015a,b; Shah et al., 2019; Sharma et al., 2019). Cattle, for example, are natural reservoirs of E. coli O157:H57, and high concentrations of this pathogen are present on feedlots and in dairy cattle manure (Berry et al., 2013, 2015; Tabe et al., 2016). Survival of E. coli O157:H7 in manure and soil environments varies widely; previous studies indicate that can survive in bovine feces from 49 to 126 days, and from 25 to 231 days in manure-amended soil depending on STEC inoculum and experimental conditions (Fremaux et al., 2008). A multi-year longitudinal experimental study conducted in the Mid-Atlantic United States identified spatio-temporal (i.e., site, year, season), agricultural (i.e., amendment type, organic or conventional, and depth of manure application), and weather factors (i.e., average daily temperature, rainfall, and initial soil moisture) contributed for the variability of the survival duration of generic E. coli and attenuated E. coli O157:H7 in manure-amended soils (Sharma et al., 2019). Temperature and precipitation influenced the likelihood of L. monocytogenes and Salmonella in vegetable farms (Strawn et al., 2013a). Moreover, on-farm studies have reported high variability of findings due to several factors (e.g., wildlife intrusion, livestock proximity, irrigation water) influencing STEC and E. coli O157:H7 prevalence in pre-harvest produce in many production environments (Jay et al., 2007; Benjamin et al., 2015; Berry et al., 2015; Weller et al., 2020). Conventional and/or controlled field trials have identified multiple factors which can affect pathogen survival and are directly related to type of livestock production, agriculture practices, soil type, and climatic factors. Therefore, on-farm multi-regional longitudinal studies are required to identify the time-variant and specific organic farming factors associated with the survival of foodborne pathogens in fresh produce grown on organic farms using untreated manure-based soil amendments. Understanding the complexity and how these factors interplay is crucial for creating a framework to support the best practices for growers to manage untreated manure. This study therefore was conducted to identify farm-level predictors for the prevalence of STEC in soils amended with untreated manure on USDA-NOP-certified farms in four US regions.

2. Materials and methods

2.1. Study design and field sample collection

A 21-month longitudinal, multi-regional field study was conducted on 19 USDA-NOP certified farms located in four USA regions (California, Minnesota, Maine, and Maryland) during two growing seasons (March 2017–December 2018). Details on the recruitment, study design and farm enrollment can be found in a previous publication (Ramos et al., 2021). Briefly, farms were enrolled based on the following criteria: (1) certified USDA-NOP fresh produce farm, (2) use of untreated manure (i.e., raw or untreated manure), (3) grow at least one of the following fresh crops: leafy greens, root vegetables, and/or fruits, (4) willingness to participate and provide biological samples (raw manure, soil, irrigation water, and fresh produce) and information about their farm management practices and location. Farms were sampled periodically from March 2017 to December 2018 using the following scheme in each growing season: (1) untreated manure, compost and soil were sampled before manure application, (2) soil samples were collected after manure incorporation and monthly for a total of seven times, (3) fresh produce, and (4) irrigation water. Sample collection and preparation are described below.

The following samples (a total of 3,260 samples) were collected from the farms during the two growing seasons: 233 untreated manure samples (cattle, horse, small ruminants, or poultry), 2,461 soil samples, 527 harvested produce samples [leafy greens (e.g., lettuce, spinach, kale), root vegetables (e.g., carrots, radishes) and/or fruits (e.g., tomatoes, cucumbers, peppers)], and 39 irrigation water samples.

2.2. Questionnaire design

A questionnaire was developed and used on each farm to obtain data on farm-level management practices and environmental characteristics. The questionnaire included: (i) farm demographics and information (e.g., farm size, number of staff, third-party certifications, marketing outlets, worker training); (ii) crop production (e.g., crop types, previous plant/soil diseases), (iii) soil characteristics and surrounding landscape (e.g., soil type, soil testing, previous use, use for livestock/poultry, surrounding landscape use), (iv) fertilizer use [e.g., manure and compost use, manure type, source, storage, application method, rate and frequency, in-house vs. commercial sources, other animal biological soil amendments, location of BSAAO relative to the crops (down/up-hill)]; (v) agricultural water (e.g., source, irrigation type, testing, irrigation frequency), and (vi) management practices (e.g., tillage practices, crop rotation, livestock/poultry in property and proximity to livestock/feedlots, integration of livestock, presence of domestic/companion animals, presence of wildlife, evidence of animal feces in fields). In addition, time-variable factors were recorded for each month visited, including intrusion of wild and domestic animals in the fields, abnormal field conditions, irrigation frequency, type of produce crop, and growth stage. The full questionnaire can be accessed on request.

2.3. Sample collection and preparation

Samples were collected and tested as previously described (Ramos et al., 2021). Farms were sampled monthly for a period of up to 180 days over two growing seasons. On each site, a total of five composite untreated manure samples was collected from manure piles using a shovel and a sterile scoop (Fisher Scientific, Hampton, NH). Before manure application (d0A, baseline), each composite sample was collected from five separate spots and different depths (not surface) in the manure pile with at least 90–100 g total per composite sample. For each manure and produce type, four composite soil samples (five subsamples each) were collected after manure incorporation (d0B) and monthly, for a total of seven times (30, 60, 90, 120, 150, and 180-days post-application (dpa) of manure) for each crop cycle as described by Ramos et al. (2021). Soil samples were collected using a stainless-steel soil core sampling probe (2.54 cm diam) disinfected with 70% ethanol between samples and 15.2 cm deep from the soil surface (Ramos et al., 2021). Agricultural water was collected once at each farm during each growing season as previously described (Ramos et al., 2021). Briefly, water samples were collected directly into a sterile Nalgene bottle (Thermo Scientific, Rochester, NY) by hand or with a sampling pole. A maximum of two irrigation water samples (1 L each) were collected from different sources (e.g., agricultural well, pond, reservoir, lagoon, or creek). All samples were collected by one of the research team members, transported or shipped to the laboratory in an insulated cooler with ice packs, and processed within 48 h post-collection.

2.4. Soil, compost, and manure sample preparation

All samples were prepared as previously described by Ramos et al. (2021) for detection, isolation, and identification using three separate enrichment schemes for non-O157 STEC, L. monocytogenes, E. coli O157:H7, Salmonella spp., and generic E. coli. Here we describe details of the enrichment and isolation of non-O157 STEC. Details on the isolation and identification of other three pathogens can be found in Ramos et al. (2021). For each manure, compost, and soil sample collected, 30 g of sample were weighed and transferred to 24 oz Whirl-Pak bags (Nasco, Modesto, CA) containing 270 ml of tryptic soy broth (TSB; 1:10 dilution), then TSB bags were manually massaged for 1 min before incubation at 25°C for 2 h, followed by a 42°C incubation for 8 h shaking at 100 rpm in a Multitron programmable shaking incubator (Eppendorf, Hauppauge, NY, USA). Then samples were held at 6°C with no shaking until used for further enrichment and isolation of non-O157 STEC.

2.5. Irrigation water sample preparation

Irrigation water samples were processed as detailed previously (Partyka et al., 2018; Ramos et al., 2021). Briefly, water samples were analyzed for bacterial indicators (i.e., generic E. coli and non-E. coli fecal coliforms) and pathogens (non-O157 STEC, E. coli O157:H7, L. monocytogenes, and Salmonella) using standard membrane filtration techniques. Water samples were vacuum-filtered through a sterile 0.45 mm nitrocellulose filter assembly.

2.6. Bacterial enrichment and isolation

2.6.1. Non-O157 STEC detection and isolation

For qualitative non-O157 STEC confirmation (Cooley et al., 2013), 1.0 ml of the initial TSB pre-enrichment broth was transferred to 9.0 ml of modified enterohemorrhagic E. coli (mEHEC) selective media (Biocontrol, Bellevue, WA, USA) tubes, which were incubated at 42°C for 12 h with agitation. Ten microliter were then streaked onto ChromSTEC (CHROMagar™, Paris, France) with an inoculation loop, and the plates were incubated at 37°C for 24 h. From the primary plates, multiple suspect non-O157 STEC colonies, based on color and morphology, were selected and sub-streaked to secondary and tertiary plates. Presumptive positives were confirmed for the presence of stx1 (348 bp) and stx2 (584 bp) genes using a traditional polymerase chain reaction (PCR) assay (Paton and Paton, 1998).

2.6.2. Generic E. coli enrichment, enumeration, and isolation

Soil and produce samples were tested for the presence and concentration of generic E. coli (Most Probable Number, MPN/g) as previously described (Atwill et al., 2015; Patterson et al., 2018). TSB was transferred from sample TSB bags (following inoculation, but prior to incubation) to TSB reservoirs (E&K Scientific, Santa Clara, CA, USA), which were serially diluted and used for enumeration of generic E. coli. Briefly, the first column of a 48-well reservoir was filled with 5 ml of the sample, followed by five columns filled with 4.5 ml TSB. Serial dilutions were carried out up to 10⁻⁶ in four replications per sample (Atwill et al., 2015; Patterson et al., 2018). Samples were incubated at 25°C for 2 h followed by 42°C for 8 h with 100 rpm shaking. Samples were then held at 6°C with no shaking. Four microliter from each dilution in the TSB reservoir were streaked onto CHROMagar ECC (CHROMagar, Paris, France) followed by incubation for 24 h at 37°C (Atwill et al., 2015). The MPN series cell densities were calculated based on dilution to extinction using an MPN Calculator (Curiale, 2004). After recording each potentially positive sample, up to four presumptive generic E. coli colonies per positive sample were selected and re-streaked onto secondary and tertiary CHROMagar ECC plates. After their respective incubations, the pure isolates were confirmed as generic E.coli using a PCR assay targeting the universal stress protein (uspA) gene (884 bp, 5′CCGATACGCTGCCAATCAGT3′, 5′ ACGCAGACCGTAGGCCAGAT 3′), as described previously (Chen and Griffiths, 1998). For each positive sample, up to four isolates were obtained and preserved in a 15% glycerol 85% TSB solution, and stored indefinitely in a −80°C freezer.

2.7. Statistical analyses

2.7.1. Data management and sources of explanatory variables for statistical analyses

The outcome of interest for analyses was non-O157 STEC in the soil after manure application (d0B) measured as a binary absence or presence. Predictor variables (five main sources) included: (1) sample data, (2) survey data, (3) HOBO data (soil moisture and temperature), (4) soil composition, and (5) weather data, with (1) to (4) sampled by our team and (5) obtained from an outside source. Analysis was later conducted in two steps, the first focusing on variable groups (1) and (2) and the second step focusing on variable groups (3) to (5). Overall, 67 variables (22 sample-level variables and 31 survey questions, transformed into 45 variables when counting dummy variables; Supplementary Table 1) were used in the first step of analysis. These variables were classified in four broad groupings: spatio-temporal (state, associated with the plot and sampling time, produce type of sampled plot); microbiological characteristics [i.e., the presence of generic E. coli and pathogenic bacteria (non-O157 STEC, L. monocytogenes, E. coli O157:H7, and Salmonella spp) in soil, manure, water]; management practices (i.e., practices for use of manure, compost, irrigation, animal presence, planted produce) and demographics (i.e., farm size, number of sites, staff). Survey variables and variables relating to bacterial presence in manure or water were aggregated at the farm-year level and field-level (for farms with more than one manure type application). Survey questions with multiple choice answers were either divided into binary dummy variables, or had multiple answers grouped into a “mixed category” if this did not represent the majority of answers. Survey variables with more than 200 missing data points were excluded from further analysis.

HOBO data (i.e., soil moisture and temperature sensors) was collected for each farm every 15 min and included soil temperature and moisture. These were aggregated daily to obtain the minimum, maximum, average, and range of temperature each day and the minimum, maximum, and average soil moisture each day. HOBO data was matched with sampling days (every 30 days) so that each sample had two sets of soil temperature and moisture variables: one for the day of sampling and one for the average of the seven days prior to the day of sampling. This data was farm- and day-specific, but not sample-specific (Supplementary Table 1). HOBO related data with missing data points (>700) due to mis-function of the sensors were excluded from the multivariable model selection.

Soil composition data was collected each year at the beginning and end of the study (d0A and 150 or 180 dpa). Common indicators assessed by the regional laboratories were considered for statistical analysis (23 in total, Supplementary Table 1), with a total of 18 variables considered for multivariable analysis.

Weather data was obtained from the National Oceanic and Atmospheric Administration (NOAA) through the R package rnoaa (Chamberlain, 2021). All weather stations within a 40 km radius of a farm were identified. Weather variables for a single farm were obtained by doing an inverse squared distance-weighted average of all surrounding stations so that closer stations had much higher weight than more distant stations. Variables included precipitation, snowfall, snow depth, temperature (average, median, minimum, maximum, range), average wind speed, and 2-s wind-gust speed. For each set of variables, three time periods were considered: weather on the day of sample collection, the average of 3 days up to sample collection, and the average of 7 days up to sample collection. Weather related variables with missing data greater than 400 data points were not considered for further multivariable analysis.

In summary, the second step of multivariable analysis used 45 variables: 18 soil variables, seven weather metrics in 21 variables and six variables from the sample and survey data of the first step (state, year, sampling day, generic E. coli concentration, soil type and soil moisture).

2.7.2. Risk analyses

To identify specific and quantifiable relationships between predictor variables and the presence of non-O157 STEC, mixed effect logistic regression was conducted in R using the lme4 package (Bates et al., 2015). For this analysis, HOBO (soil moisture and temperature), weather, and soil composition variables were standardized by subtracting the mean and dividing by the standard deviation. In all models, farm and plot ID were used as nested random effects with plot nested within a farm. First, all variables were assessed using univariable models. Then multivariable models were selected through forward and backward stepwise model selection starting from an empty model using the Akaike Information Criterion (AIC) as the selection criterion for model fit with a likelihood ratio test. Because some of the sets of variables had missing values for different observations, multivariable model selection was conducted in two data sets: a first set focusing on sample variables and survey questions, and a second set focusing on weather and soil composition data. By convention, it was decided to use a p-value of below 0.05 for a significant association and p-values between 0.05 and 0.1 for borderline significant associations.

To assess the performance of the selected multivariable models, Receiver Operating Characteristics (ROC) curves were used with the R package pROC (Robin et al., 2011). This allowed us to test the predictive performance of a model by comparing the model's fitted probabilities to the observed binary results of the outcome. The overall model performance was evaluated using the area under the curve (AUC). Then an optimal probability threshold was selected using the Youden's J statistic which maximizes both model sensitivity (SN) and specificity (SP). Above this optimal threshold, an observation is predicted as positive, and below it is predicted as negative. With the selected threshold, we can obtain model performance metrics such as SN, SP, and positive and negative predictive values (PPV and NPV). All data management and statistical analysis were conducted in R 4.0.3 (R Core Team, 2020).

3. Results

The current manuscript is focused on investigating the risk factors associated with the prevalence of STEC in soil amended with untreated manure on USDA-NOP-certified farms in four US regions, while the previous manuscript described in detail the prevalence of foodborne pathogens over time (Ramos et al., 2021). A brief description of overall prevalence is described herein as those variables were used as predictors in the models. In total, 19 farms participated in the study, of which 15 participated in both years of the study, for a total of 34 farm-years. Three farms participated in 2017 alone and a fourth in 2018 alone. Overall, we analyzed 2,150 soil samples, of which 174 (8.1%) were positive for non-O157 STEC and 101 (4.7%) were positive for L. monocytogenes. Fifteen soil samples (0.7%) were positive for Salmonella and a single sample was positive for E. coli O157:H7. The full detail of non-O157 STEC significant associations of the univariable model is described in Table 1. Of note, but not included in the broader categories described below, is the association of soil moisture with the pathogens. An increase of 10% in soil moisture was significantly associated with higher odds of positive non-O157 STEC soil samples (OR = 1.29, p-value = 0.015; Table 1). The total number of samples collected for soil composition was 330 and varied from 5 to 102 between farms (median = 8, mean = 17) or 2 to 60 per farm-year (median = 5, mean = 10). The number of weather stations within a 40 km radius of each farm varied from 1 to 231 (median = 39, mean = 53).

TABLE 1

Table 1. Univariable analysis, using a mixed-effect logistic regression model, of the factors (sample-level, time-dependent and farm questionnaire variables) that influence the likelihood of non-O157 STEC presence in soil samples amended with untreated manure in certified organic farms.

3.1. Univariable analyses models

3.1.1. Microbial presence in the soil, manure, and water

Overall, the concentration of generic E. coli (generic E. coli log₁₀ MPN+1/g of dry weight of soil) in the soil on the day of sampling (Odds Ratio, OR = 1.26; p-value < 0.001) and before manure application (d0A; OR = 1.72; p-value < 0.001) had a significant positive association with the presence of non-O157 STEC in the soil (Table 1). Conversely, higher concentrations of generic E. coli in irrigation water were associated with lower odds of positive samples for non-O157 STEC (OR = 0.49; p-value < 0.001) in the soil (Table 1). The presence of non-O157 STEC in manure (OR = 4.64, p-value < 0.001) and soil prior to manure application (d0B, baseline; OR = 2.54, p-value = 0.004) was significantly associated with the non-O157 STEC presence in soil (Table 1). Finally, the presence of other pathogens (E. coli O157, L. monocytogenes, and Salmonella spp.) in the soil had a negative association with non-O157 STEC (OR = 0.57, p-value = 0.018; Table 1).

3.1.2. Manure and management practices

Four questions from the questionnaire related to management practices for manure application (i.e., season of application, the rate of application, the number of prior years of applications, and the application method) and one for manure storage, were divided into four binary dummy variables (i.e., Yes/No for that variable). All of these variables had a significant association with non-O157 STEC in soil. Applying manure in fall (OR = 0.20, p-value = 0.009) or multiple seasons (OR = 0.38, p-value = 0.033) had significantly lower odds of non-O157 STEC compared to spring (Table 1). Having 3 to 5 years of prior manure application on the field was associated with lower odds of positive soil samples (OR = 0.24, p-value < 0.001; Table 1). Two manure application methods were associated with non-O157 STEC: surface application having increased odds of positive non-O157 STEC samples (OR = 4.70, p-value < 0.001) and grazing having lower odds (OR = 0.10, p-value < 0.001; Table 1). Manure storage was significantly associated with lower odds of non-O157 when using barn or bedding only (OR = 0.35, p-value < 0.001) compared to a manure pile (Table 1).

3.1.3. Compost management practices

Three survey questions related to compost management: source, application rate and method. The compost application method was divided into five dummy binary variables. Three variables had a significant association with the presence of non-O157 STEC in the soil. Farms with purchased compost had a significant lower odds of non-O157 STEC positive samples compared to on-farm composting (OR = 0.45, p-value = 0.037; Table 1). Compost application at a rate of 30 to 60 cubic yards per acre had significantly higher odds of a non-O157 STEC positive soil sample compared to 0 to 10 cubic yards per acre (OR = 3.42, p-value = 0.046; Table 1). Plowing and surface application of compost were both significantly associated with higher odds of positive non-O157 STEC soil samples (OR = 8.86 and 5.40 respectively, p-values < 0.001; Table 1). The three other application methods showed evidence of lower odds of non-O157 STEC positive samples, with this association being significant for disking in and trenching (OR = 0.30 and 0.07 respectively, p-values < 0.001), and borderline significant for side dressing (Table 1).

3.1.4. Presence of animals on or around the fields

Multiple survey questions were analyzed related to the presence of domestic animals (six variables) and wildlife (two variables) on or near the field. In general, variables relating to domestic animals were statistically significantly associated with non-O157 STEC positive soil samples. This was the case with five animal-related variables: field once used for livestock, presence of companion animals on the field, presence of livestock on the property, presence of domestic animals on the field, and intentional introduction of animals on the field (OR from 2.93 to 6.99 with p-values from <0.001 to 0.018; Table 1).

3.1.5. HOBO (soil temperature and moisture) and weather data

The univariable analysis of HOBO and wind variables are described in detail in Table 2. We observed that all wind variables had a significant association with non-O157 STEC, with increases in average wind speed or wind gust speed at any lag time increasing the odds of a positive sample (OR ranging from 1.23 to 1.59, p-values ranging from <0.001 to 0.046; Table 2). An increase in the temperature range on the day of sampling was associated with higher odds of a positive test (OR = 1.29, p-value = 0.021). Whereas an increase in minimum temperature on the day of sampling was associated with lower odds (OR = 0.85, p-value = 0.047; Table 2). For HOBO data, the soil moisture variable (i.e., measured at plot/field level) had no significant association with non-O157 STEC. This contrasts with the significant positive association observed with the moisture measured directly from the soil samples collected for pathogen testing (Table 1). Soil temperature variables on the day of sampling (average, minimum and maximum) were all significantly associated with presence of non-O157 STEC, whereas the 7-day averages were not. Increases in soil temperature led to a decrease in the odds of positive non-O157 STEC (OR ranging from 0.74 to 0.76, p-values from 0.015 to 0.21; Table 2). Similar, though mostly non-significant, odds ratios were observed with the ambient air temperature variables. Further univariable results for weather and soil composition data will be discussed only when contrasting with multivariable results, with details presented in Tables 2, 3.

TABLE 2

Table 2. Univariable analysis, using a mixed-effect logistic regression model, of the weather and HOBO-related factors that influence the likelihood of non-O157 STEC presence in soil samples amended with untreated manure in certified organic farms.

TABLE 3

Table 3. Univariable analysis, using a mixed-effect logistic regression model, of the soil-related variables (composition and soil health indicators) that influence the likelihood of non-O157 STEC presence in soil samples amended with untreated manure in certified organic farms.

3.2. Multivariable analyses models

3.2.1. Multivariable models—Model 1: Sample and questionnaire variables

Multivariable analysis of model 1 was performed on 1,574 complete soil samples (73% of the total). Six variables were retained in the final multivariable model including year, sampling day, and four survey questions: use of the field for livestock, presence of animal feces on the field, manure application season, and the manure application method (dummy variable). Year was strongly and significantly associated with a positive test with 2018 having much lower odds of a positive test compared to the 2017 growing season (OR = 0.05 for non-O157 STEC, p-value < 0.001; Table 4). This was comparable to univariable results (OR = 0.07 for non-O157 STEC; Table 1). Although sampling day moderately improved model fit but was not significantly associated with outcome, it was kept in the model because sampling day was one of the main exposures of interest in foodborne pathogen persistence over time. Though sampling day was not statistically significantly different from day 0, there was an increasing trend from day 90 to 180 dpa, with odds ratios increasing from 0.51 to 1.80 (Table 4). This is opposed to the univariable findings: samples collected on days 60dpa and 90dpa had significantly lower odds (OR = 0.40 and OR = 0.36) and day 180 dpa had significantly higher odds (OR = 1.72) of being non-O157 STEC positive (Table 1). Both previous use of livestock on the field and the presence of feces on the field were significantly associated with higher odds of positive samples (OR = 2.0, p-value = 0.013 for livestock and OR = 2.47, p-value = 0.001 for feces; Table 4). The result for feces was different from the univariable level where we observed a non-significant negative association with non-O157 (data not shown). For manure application season, summer had significantly higher odds of positive samples compared to spring, whereas fall had significantly lower odds (Table 4). This partly differed from the univariable results, in which summer was not significantly different from spring. Finally, the use of disking, roto-tilling, or spading had significantly lower odds of positive samples than not (OR = 0.46, p-value = 0.041; Table 4), which is the opposite of the effect observed at the univariable level where higher odds of borderline significance resulted.

TABLE 4

Table 4. Multivariable analysis, using a mixed-effect logistic regression model, of the sample-level, time-dependent and farm questionnaire factors (model 1) that influence the likelihood of non-O157 STEC presence in soil samples amended with untreated manure in certified organic farms.

3.2.2. Multivariable models—Model 2: Weather and soil composition variables

The multivariable analysis was conducted on 1,978 complete observations out of 2,150 in this model (92%). Four key sample variables were maintained in this model selection step: year, sampling day, concentration of generic E. coli in the soil, and soil moisture of samples. All four of these variables were forced into the model, based on previous literature. Another two variables from the sample and survey data were included in selection, but not forced into the model (state and soil type). In addition, one weather variable and one soil composition variable were included in the multivariate model 2 through the stepwise selection process. Year and sampling day had very similar results to those in the model from model 1 (Tables 4, 5). However, day 60 post-manure application had significantly lower odds of a positive sample in this model compared to the reference (d0B, post manure application; Table 5). An increase in both the level of generic E. coli and soil moisture were associated with increased odds of a positive test (OR = 1.25 and 1.37 respectively, p-values = 0.011; Table 5).

TABLE 5

Table 5. Multivariable analysis, using a mixed-effect logistic regression model, of the weather and soil composition factors (model 2) that influence the likelihood of non-O157 STEC presence in soil samples amended with untreated manure in certified organic farms.

The two other variables selected included the average daily snowfall for the seven days preceding sampling and soil sodium concentration. Increased snowfall led to decreased odds of a positive sample (OR = 0.60, p-value = 0.002), and increased sodium concentration was associated with increased odds of a positive sample (OR = 1.54, p-value = 0.002; Table 5). Despite so few variables being selected in the multivariable model for non-O157 STEC, nearly all soil composition variables but four (organic matter, CO₂-C, copper, and iron concentration) had a significant association with non-O157 STEC at the univariable level, with pH, aluminum and boron concentrations having a negative association and the others having a positive association (Table 3).

3.2.3. Model performance—ROC curves

Both multivariable models had similarly moderately high-performance metrics as a whole with the ROC curve analysis. The AUC was 0.83 for model 1 and 0.87 for model 2 (Table 6). Model specificity (SP) and sensitivity (SN) values varied from 0.70 to 0.87, respectively, with a higher SN in model 1 and a higher SP in model 2. In both cases, the negative predictive values (NPV) were very high, above 0.95, but the positive predictive values (PPV) were very low (0.18 in model 1 and 0.32 in model 2), due to the overall low prevalence of these pathogens in the soil samples.

TABLE 6

Table 6. Receiver operating characteristics (ROC) curve analysis results for models 1 and 2.

4. Discussion

This study is one of the first to simultaneously assess key farm management, environmental and weather factors associated with the presence of non-O157 STEC in organically managed soils amended with untreated manure on USDA-NOP certified organic farms in four regions in the USA. Several factors such as manure practices, presence of animal feces, previous land use, soil characteristics, and microbial population were associated with this pathogen. This study provides a broader foundational base of knowledge on the multitude of factors contributing to the persistence of both pathogenic non-O157 STEC and non-pathogenic in organically managed fields. Furthermore, it highlights the importance of multi-year, multi-site, and multi-regional studies to detect the significance of spatial and temporal effects. In particular, it shows that a single metric such as time interval between application of untreated manure and harvest of a crop may not be sufficient to reflect the actual food safety risks of using untreated BSAAO. Most of the published studies have been conducted in conventional agricultural settings. Multi-year and multi-site experimental trials have shown that spatio-temporal (site, season, soil moisture), agricultural (amendment type, management, depth), and weather factors (daily rainfall and average temperature) affected the duration of generic E. coli (gEc) and attenuated O157 E. coli (attO157) survival in soils amended with different soil amendments in the Mid-Atlantic USA (Sharma et al., 2019). In observational studies conducted on New York State farms, farm management and weather factors (e.g., precipitation, temperature) were associated with the prevalence of Salmonella and Listeria monocytogenes in the produce production environment (Strawn et al., 2013a,b; Weller et al., 2015).

4.1. Manure and compost management practices

Application of manure can introduce foodborne pathogens into the soil and increases the risk of soil and fresh produce contamination (Harris et al., 2013; Strawn et al., 2013b; Park et al., 2015). Management practices and environmental factors, such as the season of application, soil characteristics, and weather-related factors, can influence such presence of foodborne pathogens in the soil (Strawn et al., 2013a,b; Marine et al., 2015; Pagadala et al., 2015; Weller et al., 2015); however, the NOP requirements are based on a single metric: time-interval criteria (USDA-NOP, 2011). In the present study, foodborne pathogen presence in manure and soil (d0A, before application) and manure management-related factors (animal species source, age, storage, application manure rate, season, and method) were significantly associated with non-O157 STEC presence in the soil in the univariate model (Table 1). However, only manure application season (summer and fall) and application method were statistically significant in the final multivariable model (Table 2). Application of manure in the summer increased the odds of detecting non-O157 STEC in the soil, whereas fall application decreased the odds compared to spring application. Most of the farms apply manure during the fall, either because of state and local regulatory requirements (Midwest and East coast) (Pires et al., 2018). Moreover, fall application allows for a greater time of manure breakdown, while allowing to be in compliance with NOP required time-interval of 90–120 days between application and harvest of fresh produce (Pires et al., 2018). These findings have particular importance because it requires defined planning for the planting and harvesting of crops covered by FSMA-PSR.

Seasonal variations in the presence of foodborne pathogens in soil and contamination of fresh produce have been described in several studies (Moriarty et al., 2011; Marine et al., 2015; Sharma et al., 2019). In a Mid-Atlantic USA study, depending on site and initial soil moisture, season contributed to the survival variability of gEC and attO157 in soils amended with manure (Sharma et al., 2019). Moreover, Strawn et al. (2013b), reported that the application of manure in the prior year was associated with the risk of Salmonella-positive field samples (Strawn et al., 2013b). However, the farms in the present study applied manure every year as a soil amendment, and the manure-related factors (livestock source, application rate, storage, etc.) were not significant in the final model.

Farms applying untreated manure followed by rototilling/disking were less likely to have a non-O157 STEC-positive soil sample. Other manure incorporation methods were less commonly used by organic farmers, and most of the farms would use a combination of two or more methods (Pires et al., 2018). Entry et al. (2010) reported that the survival of fecal indicators organisms (i.e., E. coli and Enterococcus sp.) in radish in rhizosphere soil and radish roots were influenced by tillage method and depth of incorporation of dairy manure into the soil (Entry et al., 2010). Sharma et al. (2019) described that the survival of attenuated pathogens (gEc and attO157) in manure-amended soils was affected by application method (i.e., surface-applied manure, tilled-in manure), and the effect depended on the inoculum level and type of amendment, with poultry manure supporting the longest survival. In contrast, Hutchison et al. (2004) showed that the decline of three foodborne pathogens (i.e., E. coli O157:H7, Salmonella spp., and Campylobacter spp.) was significantly more rapid for livestock manure when it was left on the soil surface, potentially due to desiccation and UV effect (Hutchison et al., 2004). Other environmental conditions such as ambient temperature affected the rate of die-off of Salmonella and E. coli in soil amended with bovine manure (Natvig et al., 2002). Moreover, a recent meta-analysis study reported that temperature (soil and air combined) was the most significant factor affecting persistence of E. coli and Salmonella spp, in addition to the application methods, with manure incorporation into the soil with longer survival compared to surface application (Tran et al., 2020). The inconsistent findings among studies may show that there are factors other than application method affecting the persistence and survival of pathogens, such as microbial competition, soil characteristics, and environmental factors.

4.2. Previous use of the field and presence of feces

Previous use of the field for livestock and the presence of animal feces on the field (as reported by the farmer) were associated with a higher risk of a non-O157 STEC-positive sample in the soil. Livestock can carry foodborne pathogens intermittently, and the direct fecal deposit of feces can be a source of the pathogens, which, depending on the pathogen, can survive in the soil for long periods (Alegbeleye et al., 2018). Adjacent land use for cow-calf rearing and the presence of feral pigs were implicated in the 2007 spinach outbreak due to E. coli O157:H7 in Central Coast California (Jay et al., 2007). More recently, the 2018 multi-state E. coli O157:H7 outbreak associated with romaine lettuce and potential contamination of water due to proximity to a large animal feeding operation highlights the need to investigate the proximity of livestock operations to fresh produce fields (Waltenburg et al., 2021). Berry et al. (2015, 2019) reported that 400 feet may not be adequate to limit the transmission of E. coli O157:H7 to produce crops close to a Concentrated Animal Feeding Operation (CAFO), and flies may have a role in leafy greens contamination (Berry et al., 2015, 2019). Actually, the prevalence of enterohemorrhagic E. coli was highest on fresh produce fields near areas suitable for livestock grazing on California Central Coast farms (Karp et al., 2015).

Integration of crop-livestock has been increasing in different regions in the US. Despite many benefits from the presence of livestock, it increases the risk of cross-contamination of vegetable and nut crops (Salaheen et al., 2015; Patterson et al., 2018). A study conducted in sustainable farming systems in Tennessee and North Carolina suggested the potential transmission of antimicrobial-resistant non-O157 E. coli between animal feeding operations and fresh produce (Glaize et al., 2020). In order to reduce contamination, it is important to assess the risk of contamination from the surrounding landscape and adjacent land use, such as proximity of livestock operations and potential wildlife intrusions. As part of the Leafy Greens STEC Action Plan, FDA is conducting a multi-year California longitudinal study to assess the pathogen dynamics in the environment, particularly landscape and adjacent land use effects on contamination (FDA, 2022).

Presence of fecal material increased the odds of non-O157 STEC-positive samples in the soil. Although no differentiation in the analysis was made between fecal material from domestic animals and wildlife, the majority of those observations were linked to wildlife presence (research team observations). Wildlife such as birds, reptiles, feral pigs, and deer are well-known carriers of foodborne pathogens such as Salmonella, Campylobacter, and pathogenic E. coli (Jay et al., 2007; Jay-Russell et al., 2012; Gorski et al., 2013; Alegbeleye et al., 2018; Navarro-Gonzalez et al., 2020). Measuring the direct risks of wildlife intrusion and fresh produce production contamination is complex; however, several outbreaks have been linked to wildlife intrusion. Deer were identified as the causal source of an E. coli O157:H7 outbreak in strawberries in Oregon (Laidler et al., 2013), and cranes were the environmental source for a campylobacterosis outbreak in peas (Gardner et al., 2011). The FSMA-PSR includes language for prevention of contamination and risk mitigation; however, this is based on individual farm assessments, which may vary from grower to grower and may be landscape-specific (FDA, 2015). Determining adequate no-harvest buffers around in-field fecal material is necessary in order to avoid the contamination of fresh produce by foodborne pathogens (Salmonella, E. coli, and E. coli O157:H7), which depends on fecal sources, irrigation, and splash events (Atwill et al., 2015; Weller et al., 2017; Topalcengiz et al., 2020). An experimental study conducted in the Northeastern United States showed E. coli levels on lettuce and lettuce-feces distance were negatively correlated, when using a buffer smaller or equal to 1 meter during foliar regulation (Weller et al., 2017). The current recommendations of the California Leafy Greens Market Agreement (LGMA) if there is evidence of fecal material presence is not to harvest any crop within a minimum of five-foot radius buffer distance from the contamination site (LGMA, 2021). Therefore, evaluation is needed for the risk of wildlife intrusion, either by fecal material presence or crop destruction, and implementation of mitigation strategies, such as buffer zones and crop destruction.

4.3. Soil: Microbial population and soil characteristics

A multitude of biotic and abiotic factors affect the persistence of foodborne pathogens in soil such as soil type (e.g., clay, sandy), soil composition (e.g., organic matter, moisture, nutrients), soil temperature, and soil texture (Ingham et al., 2005; Alegbeleye et al., 2018). The effect of these factors may be altered by agronomic practices and environmental factors, such as precipitation and ambient temperature (Entry et al., 2010; Sharma et al., 2019; Pang et al., 2020). In the present study, the presence of generic E. coli, soil moisture, and sodium were associated with likelihood of non-O157 presence in the manure-amended soil. In model 2, generic E. coli was significantly associated with the presence of non-O157 in the soil. Given that non-O157 STEC would contribute to the generic E. coli, this association would be expected, however, we did not quantify the pathogen, and direct correlation was not assessed. Generic E. coli may be a good predictor of non-O157 STEC presence and a fecal contamination indicator. Because detection of STEC and E. coli O157:H7 in soil is infrequent, and due to the prohibitive use of pathogenic strains in privately operated farms in experimental studies, generic E. coli has been widely used as a proxy for fecal contamination in studies investigating the survival of pathogenic E. coli in soil amended with untreated animal manure in produce fields (Reed-Jones et al., 2016; Sharma et al., 2016, 2019; Patterson et al., 2018). A recent meta-analysis reported that temperature (soil and air combined) was the most prominent factor affecting the persistence of E. coli in manure-amended soils (Tran et al., 2020). Moreover, Sharma et al. (2019) reported that moisture content was the individual factor that accounts for the greatest variability in survival duration of generic E. coli in manure-amended soils in the Mid-Atlantic USA (Sharma et al., 2019). In the current study, soil moisture was significantly associated with the presence of non-O157 STEC in the soil in the final multivariable model (model 2), and the odds of identifying a non-O157 STEC positive sample increased by 1.37 for each 10% in the soil moisture. This is in disagreement with previous studies (Park et al., 2016; Sharma et al., 2016). Lower initial soil moisture content (11.2 to 12.1%) supported longer durations of survival for certain seasons and sites in a Mid-Atlantic study (Sharma et al., 2019). Soil moisture depends on precipitation, with irrigation being crop-specific and crop-maturity–related. In the present study, we did not record irrigation events, which were adjusted according to the crop; instead, we used soil moisture at the sample level, accounting for between-sample variability which combines weather factors (precipitation and temperature) and irrigation, because it was measured at soil-sample level, as opposed to the one unique measurement at plot level by the moisture sensor (at field level measured by HOBO).

4.4. Weather and time-related factors

Interestingly, in this study, multiple weather-related factors (temperature, precipitation, wind, and snowfall) at different lag times were significantly associated with non-O157 STEC presence in the univariable analysis, but only snowfall was retained in the final multivariable analysis (model 2) with lower odds of a non-O157:H7 STEC positive soil sample. These findings contrast with other studies on the association between prevalence of foodborne pathogens in the fresh produce farm environment and weather factors. Temperature and precipitation influenced the likelihood of L. monocytogenes and Salmonella in fruit and vegetable farms from New York State (Strawn et al., 2013a). Events such as rain and precipitation were associated with the presence of Listeria species and L. monocytogenes in New York State farm environments (Weller et al., 2015). In both models, there was lower likelihood of identifying a non-O157 STEC positive soil sample in the second year of the study, even accounting for other factors. This is in agreement with inoculation experimental studies, in which spatio-temporal factors (site, year, and season) influenced survival duration of generic E. coli and attenuated O157 E. coli more than weather effects (i.e., average daily temperature and rainfall), with the year being the greatest contributor to variability (Sharma et al., 2019). This shows the importance of multi-year and multi-site studies over several growing seasons when investigating foodborne pathogen survival and persistence in soil amended with untreated manure.

Sampling day had very similar results in both models. There was a trend that the odds of a positive sample were lower up to 90 days post-manure application (compared to d0B), but not statistically significant, with the exception of sampling day 60 post-manure application. Interestingly, this trend changed direction after 120 days post-application, although not statistically significant. Part of the explanation could be due to the fact that the total number of samples per farm started to decrease after this time point and most of the farms had finished harvesting produce, only keeping the plots for the research sampling. The presence of vegetable decay and potential intrusion of wildlife could have contributed to this shift as well as to the potential introduction of foodborne pathogens. Certified organic farms follow USDA-NOP standards for wait-times (90/120 days) between the application of raw manure and crop harvest. The multivariable models show that the odds of finding a non-O157 STEC positive soil sample decreases with time. The time-interval as a mitigation strategy is based on the assumption that an appropriate amount of time has passed to reduce the risks of foodborne pathogen survival in the soil (Phan-Thien et al., 2020). However, in the present study, several on-farm practices (i.e., manure management), soil factors, and weather contributed to the non-O157 STEC presence in the soil after the application of untreated manure. Understanding how these factors interact and what the key time-variant factors are is crucial to creating a customized risk-based assessment of the farm and implementing mitigation practices for the reduction of farm environment contamination of foodborne pathogens when untreated manure is used as a soil amendment.

There are some limitations in observational studies and predictive models to assess the risks factors associated with the likelihood of pathogen presence in soil amended with untreated manure. First, we were unable to directly link the presence of non-O157 STEC in the manure and soil, as it was not part of the current study to investigate genetic relatedness for the isolates; however, no association was found between the presence of pathogens in manure and soil in the final models. Another limitation is that the study was conducted on commercial farms without interventions, in which the exact contamination load in the manure was unknown (as opposed to inoculation trials). Information collected in observational studies with multiple variables is relevant as representing the natural stage and diversity of agricultural environments, which allows us to investigate simultaneously different predictors. Moreover, data collection suffered from multiple issues that could potentially introduce bias. First, there was a large number of missing data for a number of variables. Some of these were removed but we chose to retain others due to their biological relevance to the study objectives. This particularly affected multivariable model 1 potentially leading to a loss of some external validity for this model. Next, different data groups were sampled with different temporal resolutions (sampling day vs. once or twice a year) and spatial resolutions (farm vs. plot). This was partially accounted for by using random effects but could still lead to some bias, as well as, causing a loss of analytical power due to the need to repeat identical values of predictors with lower resolution to match the resolution of the outcome variable. Finally, some variables (notably soil composition) suffered from an unbalanced sampling frame with some farms having much less samples collected, leading to the need to aggregate values at the farm level with a loss of resolution for these variables.

5. Conclusions

This study provides science-based information to identify potential risk factors influencing foodborne pathogen persistence in pre-harvest produce production environments for soils amended with raw manure on organic production systems. Results from this study provide multi-regional baseline data relating to current NOP wait-time stipulations and thus contribute to building a framework for risk mitigation strategies to reduce pathogen persistence in NOP-certified farming systems using untreated manure that could contribute to contamination of fresh produce that is typically eaten raw. These factors may be directly related to the type of livestock production and agriculture practices, soil type, and weather that vary between regions and states.

Understanding the complexity and interactions of on-farm factors is crucial for development of customized risk assessments and tools to promote best practices for growers to use in managing the use of raw manure-based soil amendments.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author contributions

AP, MJ-R, PM, PP, MH, JS, and BM-L conceived the project, designed the study, and wrote the grant. AP, PM, FH, PP, and MH coordinated the efforts between the CA, MN, MD, and ME teams. AP oversaw the day-to-day aspects of the project. AP, MJ-R, and PM oversaw the laboratory work. TR, AK, VH, and PA carried out the field work and sampling collection. TR, VH, and PA carried out the laboratory work. AP, JB, and BM-L developed the data analysis plan, which JB implemented. AP, TR, JB, and VH wrote the first draft of the manuscript. All authors contributed to manuscript revision, read, and approved the final manuscript.

Funding

This work was funded by the Organic Agriculture Research and Extension Initiative (OREI), National Institute of Food and Agriculture, U.S. Department of Agriculture, under award number 2016-51300-25724.

Acknowledgments

We acknowledge James Stover, Carolyn Chandler, Amanda Taylor, Zoe Scholar, Deonnie Huang, Rachel Hawkins, Amy Patamakomol, and Ka Man Woo for assistance in the laboratory and field work as well as Tracy Misiewicz from The Organic Center for assistance with farmer recruitment. We would like to thank all the participating farms.

Conflict of interest

JS and AS were employed by The Organic Center.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fsufs.2023.1125996/full#supplementary-material

References

Alegbeleye, O. O., Singleton, I., and Sant'Ana, A. S. (2018). Sources and contamination routes of microbial pathogens to fresh produce during field cultivation: a review. Food Microbiol. 73, 177–208. doi: 10.1016/j.fm.2018.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Atwill, E. R., Chase, J. A., Oryang, D., Bond, R. F., Koike, S. T., Cahn, M. D., et al. (2015). Transfer of Escherichia coli O157:H7 from simulated wildlife scat onto romaine lettuce during foliar irrigation. J. Food Protect. 78, 240–247. doi: 10.4315/0362-028X.JFP-14-277

PubMed Abstract | CrossRef Full Text | Google Scholar

Bates, D., Machler, M., Bolker, B. M., and Walker, S. C. (2015). Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48. doi: 10.18637/jss.v067.i01

CrossRef Full Text | Google Scholar

Benjamin, L. A., Jay-Russell, M. T., Atwill, E. R., Cooley, M. B., Carychao, D., Larsen, R. E., et al. (2015). Risk factors for Escherichia coli O157 on beef cattle ranches located near a major produce production region. Epidemiol. Infect. 143, 81–93. doi: 10.1017/S0950268814000521

PubMed Abstract | CrossRef Full Text | Google Scholar

Berry, E. D., Millner, P. D., Wells, J. E., Kalchayanand, N., and Guerini, M. N. (2013). Fate of naturally occurring Escherichia coli O157:H7 and other zoonotic pathogens during minimally managed bovine feedlot manure composting processes. J. Food Protect. 76, 1308–1321. doi: 10.4315/0362-028X.JFP-12-364

PubMed Abstract | CrossRef Full Text | Google Scholar

Berry, E. D., Wells, J. E., Bono, J. L., Woodbury, B. L., Kalchayanand, N., Norman, K. N., et al. (2015). Effect of proximity to a cattle feedlot on Escherichia coli O157:H7 contamination of leafy greens and evaluation of the potential for airborne transmission. Appl. Environ. Microbiol. 81, 1101–1110. doi: 10.1128/AEM.02998-14

PubMed Abstract | CrossRef Full Text | Google Scholar

Berry, E. D., Wells, J. E., Durso, L. M., Friesen, K. M., Bono, J. L., and Suslow, T. V. (2019). Occurrence of Escherichia coli O157:H7 in pest flies captured in leafy greens plots grown near a beef cattle feedlot. J. Food Protect. 82, 1300–1307. doi: 10.4315/0362-028X.JFP-18-601

PubMed Abstract | CrossRef Full Text | Google Scholar

Chamberlain, S. (2021). rrnoaa: ‘NOAA’ Weather Data from R. R. R package version 1.3.0 (CRAN).

Google Scholar

Chen, J., and Griffiths, M. W. (1998). PCR differentiation of Escherichia coli from other gram-negative bacteria using primers derived from the nucleotide sequences flanking the gene encoding the universal stress protein. Lett. Appl. Microbiol. 27, 369–371. doi: 10.1046/j.1472-765X.1998.00445.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Cooley, M. B., Jay-Russell, M., Atwill, E. R., Carychao, D., Nguyen, K., Quinones, B., et al. (2013). Development of a robust method for isolation of Shiga toxin-positive Escherichia coli (STEC) from fecal, plant, soil and water samples from a leafy greens production region in California. PloS ONE. 8, e65716. doi: 10.1371/journal.pone.0065716

PubMed Abstract | CrossRef Full Text | Google Scholar

Curiale, M. (2004). MPN Calculator. Available online at: http://www.i2workout.com/mcuriale/mpn/index.html (accessed July 25, 2019).

Google Scholar

Entry, J. A., Bjorneberg, D. L., and Verwey, S. (2010). Influence of tillage and daily manure application on the survival of bacterial pathogens indicators in soil and on radish. Appl. Environ. Soil Sci. 2010, 973925. doi: 10.1155/2010/973925

CrossRef Full Text | Google Scholar

Erickson, M. C., Liao, J., Ma, L., Jiang, X., and Doyle, M. P. (2014). Thermal and nonthermal factors affecting survival of Salmonella and Listeria monocytogenes in animal manure-based compost mixtures. J. Food Protect. 77, 1512–1518. doi: 10.4315/0362-028X.JFP-14-111

PubMed Abstract | CrossRef Full Text | Google Scholar

Erickson, M. C., Smith, C., Jiang, X. P., Flitcroft, I. D., and Doyle, M. P. (2015a). Manure source and age affect survival of zoonotic pathogens during aerobic composting at sublethal temperatures. J. Food Protect. 78, 302–310. doi: 10.4315/0362-028X.JFP-14-288

PubMed Abstract | CrossRef Full Text | Google Scholar

Erickson, M. C., Smith, C., Jiang, X. P., Flitcroft, I. D., and Doyle, M. P. (2015b). Survival of Salmonella or Escherichia coli O157:H7 during holding of manure-based compost mixtures at sublethal temperatures as influenced by the carbon amendment. J. Food Protect. 78, 248–255. doi: 10.4315/0362-028X.JFP-14-289

PubMed Abstract | CrossRef Full Text | Google Scholar

FDA (2015). Standards for the Growing, Harvesting, Packing, and Holding of Produce for Human Consumption. Food and Drug Administration. Final rule. Federal Register Notice. Rome: FDA, 74547–74568.

Google Scholar

FDA (2018). Standards for the Growing, Harvesting, Packing, and Holding of Produce for Human Consumption: Guidance for Industry. Draft guidance. Federal Register Notice. Rome: FDA 54888–54891.

Google Scholar

FDA (2019). Environmental Assessment of Factors Potentially Contributing to the Contamination of Romaine Lettuce Implicated in a Multi-state Outbreak of E. coli O157:H7. Available online at: https://www.fda.gov/food/outbreaks-foodborne-illness/environmental-assessment-factors-potentially-contributing-contamination-romaine-lettuce-implicated#references (accessed November 28, 2022).

Google Scholar

FDA (2022). Leafy Greens STEC Action Plan. Available online at: https://www.fda.gov/food/foodborne-pathogens/leafy-greens-stec-action-plan (accessed February 9, 2023).

Google Scholar

Fremaux, B., Prigent-Combaret, C., and Vernozy-Rozand, C. (2008). Long-term survival of Shiga toxin-producing Escherichia coli in cattle effluents and environment: an updated review. Vet. Microbiol. 132, 1–18. doi: 10.1016/j.vetmic.2008.05.015

PubMed Abstract | CrossRef Full Text | Google Scholar

Gardner, T. J., Fitzgerald, C., Xavier, C., Klein, R., Pruckler, J., Stroika, S., et al. (2011). Outbreak of Campylobacteriosis associated with consumption of raw peas. Clin. Infect. Dis. 53, 26–32. doi: 10.1093/cid/cir249

PubMed Abstract | CrossRef Full Text | Google Scholar

Glaize, A., Gutierrez-Rodriguez, E., Hanning, I., Díaz-Sánchez, S., Gunter, C., van Vliet, A. H. M., et al. (2020). Transmission of antimicrobial resistant non-O157 Escherichia coli at the interface of animal-fresh produce in sustainable farming environments. Int. J. Food Microbiol. 319, 108472. doi: 10.1016/j.ijfoodmicro.2019.108472

PubMed Abstract | CrossRef Full Text | Google Scholar

Gorski, L., Jay-Russell, M. T., Liang, A. S., Walker, S., Bengson, Y., Govoni, J., et al. (2013). Diversity of pulsed-field gel electrophoresis pulsotypes, serovars, and antibiotic resistance among Salmonella isolates from wild amphibians and reptiles in the California Central Coast. Foodborne Pathog. Dis. 10, 540–548. doi: 10.1089/fpd.2012.1372

PubMed Abstract | CrossRef Full Text | Google Scholar

Harris, L. J., Berry, E. D., Blessington, T., Erickson, M., Jay-Russell, M., Jiang, X. P., et al. (2013). A framework for developing research protocols for evaluation of microbial hazards and controls during production that pertain to the application of untreated soil amendments of animal origin on land used to grow produce that may be consumed raw. J. Food Protect. 76, 1062–1084. doi: 10.4315/0362-028X.JFP-13-007

PubMed Abstract | CrossRef Full Text | Google Scholar

Haumann, B. F. (2017). “Organic continues to set records in the United States,” in The World of Organic Agriculture. Statistics and Emerging Trends 2017, eds H. Willer, and J. Lernoud (Bonn: Research Institute of Organic Agriculture (FiBL), Frick, and IFOAM - Organics International), 260–263.

Google Scholar

Hutchison, M. L., Walters, L. D., Avery, S. M., Munro, F., and Moore, A. (2005). Analyses of livestock production, waste storage, and pathogen levels and prevalences in farm manures. Appl. Environ. Microbiol. 71, 1231–1236. doi: 10.1128/AEM.71.3.1231-1236.2005

PubMed Abstract | CrossRef Full Text | Google Scholar

Hutchison, M. L., Walters, L. D., Moore, A., Crookes, K. M., and Avery, S. M. (2004). Effect of length of time before incorporation on survival of pathogenic bacteria present in livestock wastes applied to agricultural soil. Appl. Environ. Microbiol. 70, 5111–5118. doi: 10.1128/AEM.70.9.5111-5118.2004

PubMed Abstract | CrossRef Full Text | Google Scholar

Ingham, S. C., Fanslau, M. A., Engel, R. A., Breuer, J. R., Wright, T. H., Reith-Rozelle, J. K., et al. (2005). Evaluation of fertilization-to-planting and fertilization-to-harvest intervals for safe use of noncomposted bovine manure in Wisconsin vegetable production. J. Food Protect. 68, 1134–1142. doi: 10.4315/0362-028X-68.6.1134

PubMed Abstract | CrossRef Full Text | Google Scholar

Jay, M. T., Cooley, M., Carychao, D., Wiscomb, G. W., Sweitzer, R. A., Crawford-Miksza, L., et al. (2007). Escherichia coli O157:H7 in feral swine near spinach fields and cattle, Central California Coast. Emerg. Infect. Dis. 13, 1908–1911. doi: 10.3201/eid1312.070763

PubMed Abstract | CrossRef Full Text | Google Scholar

Jay-Russell, M. T., Bates, A., Harden, L., Miller, W. G., and Mandrell, R. E. (2012). Isolation of Campylobacter from feral swine (Sus scrofa) on the ranch associated with the 2006 Escherichia coli O157:H7 spinach outbreak investigation in California. Zoonoses Public Health. 59, 314–319. doi: 10.1111/j.1863-2378.2012.01465.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Karp, D. S., Gennet, S., Kilonzo, C., Partyka, M., Chaumont, N., Atwill, E. R., et al. (2015). Comanaging fresh produce for nature conservation and food safety. Proc. Natl. Acad. Sci. USA. 112, 11126–11131. doi: 10.1073/pnas.1508435112

PubMed Abstract | CrossRef Full Text | Google Scholar

Laidler, M. R., Tourdjman, M., Buser, G. L., Hostetler, T., Repp, K. K., Leman, R., et al. (2013). Escherichia coli O157:H7 infections associated with consumption of locally grown strawberries contaminated by deer. Clin. Infect. Dis. 57, 1129–1134. doi: 10.1093/cid/cit468

PubMed Abstract | CrossRef Full Text | Google Scholar

LGMA (2021). California LGMA, Commodity Specific Food Safety Guidelines for Production and Harvest of Lettuce and Leafy Greens. Available online at: https://lgma.ca.gov/lgma-connect (accessed February 9, 2023).

Google Scholar

Luna, S., Krishnasamy, V., Saw, L., Smith, L., Wagner, J., Weigand, J., et al. (2018). Outbreak of E. coli O157:H7 infections associated with exposure to animal manure in a rural community—Arizona and Utah, June–July 2017. MMWR-Morbid. Mortal. W. 67, 659–662. doi: 10.15585/mmwr.mm6723a2

PubMed Abstract | CrossRef Full Text | Google Scholar

Marine, S. C., Pagadala, S., Wang, F., Pahl, D. M., Melendez, M. V., Kline, W. L., et al. (2015). The growing season, but not the farming system, is a food safety risk determinant for leafy greens in the Mid-Atlantic region of the United States. Appl. Environ. Microbiol. 81, 2395–2407. doi: 10.1128/AEM.00051-15

PubMed Abstract | CrossRef Full Text | Google Scholar

Moriarty, E. M., Mackenzie, M. L., Karki, N., and Sinton, L. W. (2011). Survival of Escherichia coli, Enterococci, and Campylobacter spp. in sheep feces on pastures. Appl. Environ. Microbiol. 77, 1797–1803. doi: 10.1128/AEM.01329-10

PubMed Abstract | CrossRef Full Text | Google Scholar

Natvig, E. E., Ingham, S. C., Ingham, B. H., Cooperband, L. R., and Roper, T. R. (2002). Salmonella enterica serovar Typhimurium and Escherichia coli contamination of root and leaf vegetables grown in soils with incorporated bovine manure. Appl. Environ. Microbiol. 68, 2737–2744. doi: 10.1128/AEM.68.6.2737-2744.2002

PubMed Abstract | CrossRef Full Text | Google Scholar

Navarro-Gonzalez, N., Wright, S., Aminabadi, P., Gwinn, A., Suslow, T. V., Jay-Russell, M. T., et al. (2020). Carriage and subtypes of foodborne pathogens identified in wild birds residing near agricultural lands in California: a repeated cross-sectional study. Appl. Environ. Microbiol. 86, e01678. doi: 10.1128/AEM.01678-19

PubMed Abstract | CrossRef Full Text | Google Scholar

Olaimat, A. N., and Holley, R. A. (2012). Factors influencing the microbial safety of fresh produce: a review. Food Microbiol. 32, 1–19. doi: 10.1016/j.fm.2012.04.016

PubMed Abstract | CrossRef Full Text | Google Scholar

Pagadala, S., Marine, S. C., Micallef, S. A., Wang, F., Pahl, D. M., Melendez, M. V., et al. (2015). Assessment of region, farming system, irrigation source and sampling time as food safety risk factors for tomatoes. Int. J. Food Microbiol. 196, 98–108. doi: 10.1016/j.ijfoodmicro.2014.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Pang, H., Mokhtari, A., Chen, Y., Oryang, D., Ingram, D. T., Sharma, M., et al. (2020). A predictive model for survival of Escherichia coli O157:H7 and generic E. coli in soil amended with untreated animal manure. Risk Anal. 40, 1367–1382. doi: 10.1111/risa.13491

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, S., Navratil, S., Gregory, A., Bauer, A., Srinath, I., Szonyi, B., et al. (2015). Multifactorial effects of ambient temperature, precipitation, farm management, and environmental factors determine the level of generic Escherichia coli contamination on preharvested spinach. Appl. Environ. Microbiol. 81, 2635–2650. doi: 10.1128/AEM.03793-14

PubMed Abstract | CrossRef Full Text | Google Scholar

Park, Y., Pachepsky, Y., Shelton, D., Jeong, J., and Whelan, G. (2016). Survival of manure-borne Escherichia coli and fecal coliforms in soil: temperature dependence as affected by site-specific factors. J. Environ. Qual. 45, 949–957. doi: 10.2134/jeq2015.08.0427

PubMed Abstract | CrossRef Full Text | Google Scholar

Partyka, M. L., Bond, R. F., Chase, J. A., and Atwill, E. R. (2018). Spatial and temporal variability of bacterial indicators and pathogens in six California reservoirs during extreme drought. Water Res. 129, 436–446. doi: 10.1016/j.watres.2017.11.038

PubMed Abstract | CrossRef Full Text | Google Scholar

Paton, A. W., and Paton, J. C. (1998). Detection and characterization of Shiga toxigenic Escherichia coli by multiplex PCR assays for stx(1), stx(2), eaeA, enterohemorrhagic E-coli hlyA, rfb(O111), and rfb(O157). J. Clin. Microbiol. 36, 598–602. doi: 10.1128/JCM.36.2.598-602.1998

PubMed Abstract | CrossRef Full Text | Google Scholar

Patterson, L., Navarro-Gonzalez, N., Jay-Russell, M. T., Aminabadi, P., Antaki-Zukoski, E., and Pires, A. F. A. (2018). Persistence of Escherichia coli in the soil of an organic mixed crop-livestock farm that integrates sheep grazing within vegetable fields. Zoonoses Public Health. 65, 887–896. doi: 10.1111/zph.12503

PubMed Abstract | CrossRef Full Text | Google Scholar

Phan-Thien, K. Y., Metaferia, M. H., Bell, T. L., Bradbury, M. I., Sassi, H. P., Van Ogtrop, F. F., et al. (2020). Effect of soil type and temperature on survival of Salmonella enterica in poultry manure-amended soils. Lett. Appl. Microbiol. 71, 210–217. doi: 10.1111/lam.13302

PubMed Abstract | CrossRef Full Text | Google Scholar

Pires, A. F. A., Millner, P. D., Baron, J., and Jay-Russell, M. T. (2018). Assessment of current practices of organic farmers regarding biological soil amendments of animal origin in a multi-regional U.S. study. Food Protect. Trends 38, 347–362.

Google Scholar

R Core Team (2020). R: A Language and Environment for Statistical Computing. Vienna: R Foundation for Statistical Computing.

Google Scholar

Ramos, T. D. M., Jay-Russell, M. T., Millner, P. D., Baron, J. N., Stover, J., Pagliari, P., et al. (2021). Survival and persistence of foodborne pathogens in manure-amended soils and prevalence on fresh produce in certified organic farms: a multi-regional baseline analysis. Front. Sustain. Food Syst. 5, 674767. doi: 10.3389/fsufs.2021.674767

CrossRef Full Text | Google Scholar

Rana, J., and Paul, J. (2017). Consumer behavior and purchase intention for organic food: a review and research agenda. J. Retailing Consum. Serv. 38, 157–165. doi: 10.1016/j.jretconser.2017.06.004

CrossRef Full Text | Google Scholar

Reed-Jones, N. L., Marine, S. C., Everts, K. L., and Micallef, S. A. (2016). Effects of cover crop species and season on population dynamics of Escherichia coli and Listeria innocua in soil. Appl. Environ. Microbiol. 82, 1767–1777. doi: 10.1128/AEM.03712-15

PubMed Abstract | CrossRef Full Text | Google Scholar

Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., et al. (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 12, 77. doi: 10.1186/1471-2105-12-77

PubMed Abstract | CrossRef Full Text | Google Scholar

Rosen, C. J., and Allan, D. L. (2007). Exploring the benefits of organic nutrient sources for crop production and soil quality. Horttechnology 17, 422–430. doi: 10.21273/HORTTECH.17.4.422

CrossRef Full Text | Google Scholar

Rosen, C. J., and Bierman, P. M. (2005). Nutrient Management for Fruit and Vegetable Crop Production: Using Manure and Compost as Nutrient Sources for Vegetable Crops. University of Minnesota. Available onlne at: https://hdl.handle.net/11299/200639 (accessed November 28, 2022).

Google Scholar

Salaheen, S., Chowdhury, N., Hanning, I., and Biswas, D. (2015). Zoonotic bacterial pathogens and mixed crop-livestock farming. Poultry Sci. 94, 1398–1410. doi: 10.3382/ps/peu055

PubMed Abstract | CrossRef Full Text | Google Scholar

Scallan, E., Hoekstra, R. M., Angulo, F. J., Tauxe, R. V., Widdowson, M. A., Roy, S. L., et al. (2011). Foodborne illness acquired in the United States-major pathogens. Emerg. Infect. Dis. 17, 7–15. doi: 10.3201/eid1701.P11101

PubMed Abstract | CrossRef Full Text | Google Scholar

Shah, M. K., Bradshaw, R., Nyarko, E., Millner, P. D., Neher, D., Weicht, T., et al. (2019). Survival and growth of wild-type and rpoS-deficient Salmonella Newport strains in soil extracts prepared with heat-treated poultry pellets. J. Food Protect. 82, 501–506. doi: 10.4315/0362-028X.JFP-18-465

PubMed Abstract | CrossRef Full Text | Google Scholar

Sharma, M., Millner, P. D., Hashem, F., Camp, M., Whyte, C., Graham, L., et al. (2016). Survival and persistence of nonpathogenic Escherichia coli and attenuated Escherichia coli O157:H7 in soils amended with animal manure in a greenhouse environment. J. Food Protect. 79, 913–921. doi: 10.4315/0362-028X.JFP-15-421

PubMed Abstract | CrossRef Full Text | Google Scholar

Sharma, M., Millner, P. D., Hashem, F., Vinyard, B. T., East, C. L., Handy, E. T., et al. (2019). Survival of Escherichia coli in manure-amended soils is affected by spatiotemporal, agricultural, and weather factors in the Mid-Atlantic United States. Appl. Environ. Microbiol. 85, e02392. doi: 10.1128/AEM.02392-18

PubMed Abstract | CrossRef Full Text | Google Scholar

Sinton, L. W., Braithwaite, R. R., Hall, C. H., and Mackenzie, M. L. (2007). Survival of indicator and pathogenic bacteria in bovine feces on pasture. Appl. Environ. Microbiol. 73, 7917–7925. doi: 10.1128/AEM.01620-07

PubMed Abstract | CrossRef Full Text | Google Scholar

Strawn, L. K., Fortes, E. D., Bihn, E. A., Nightingale, K. K., Grohn, Y. T., Worobo, R. W., et al. (2013a). Landscape and meteorological factors affecting prevalence of three food-borne pathogens in fruit and vegetable farms. Appl. Environ. Microbiol. 79, 588–600. doi: 10.1128/AEM.02491-12

PubMed Abstract | CrossRef Full Text | Google Scholar

Strawn, L. K., Grohn, Y. T., Warchocki, S., Worobo, R. W., Bihn, E. A., and Wiedmann, M. (2013b). Risk factors associated with Salmonella and Listeria monocytogenes contamination of produce fields. Appl. Environ. Microbiol. 79, 7618–7627. doi: 10.1128/AEM.02831-13

PubMed Abstract | CrossRef Full Text | Google Scholar

Tabe, N. N., Rahman, S., Tabe, E. S., Doetkott, D., and Khaitsa, M. L. (2016). Prevalence of Escherichia coli and Salmonella in runoff of two cattle feedlots in North Dakota. Food Protect. Trends. 36, 33–42.

Google Scholar

Topalcengiz, Z., Spanninger, P. M., Jeamsripong, S., Persad, A. K., Buchanan, R. L., Saha, J., et al. (2020). Survival of Salmonella in various wild animal feces that may contaminate produce. J. Food Protect. 83, 651–660. doi: 10.4315/0362-028X.JFP-19-302

PubMed Abstract | CrossRef Full Text | Google Scholar

Tran, D. T. Q., Bradbury, M. I., Ogtrop, F. F. V., Bozkurt, H., Jones, B. J., and McConchie, R. (2020). Environmental drivers for persistence of Escherichia coli and Salmonella in manure-amended soils: a meta-analysis. J. Food Protect. 83, 1268–1277. doi: 10.4315/0362-028X.JFP-19-460

PubMed Abstract | CrossRef Full Text | Google Scholar

Turner, K., Moua, C. N., Hajmeer, M., Barnes, A., and Needham, M. (2019). Overview of leafy greens-related food safety incidents with a California link: 1996 to 2016. J. Food Protect. 82, 405–414. doi: 10.4315/0362-028X.JFP-18-316

PubMed Abstract | CrossRef Full Text | Google Scholar

USDA-AMS (2000). National Organic Program, Final rule. Federal Register. Washington, DC: United States Department of Agriculture, A.M.S. 1–50.

Google Scholar

USDA-NOP (2011). National Organic Program Handbook: Guidance and Instructions for Accredited Certifying Agents and Certified Operations. Washington, DC: United States Department of Agriculture, A.M.S., National Organic Program, Guidance: Processed Animal Manures in Organic Crop Production, 1–3.

Google Scholar

Waltenburg, M. A., Schwensohn, C., Madad, A., Seelman, S. L., Peralta, V., Koske, S. E., et al. (2021). Two multistate outbreaks of a reoccurring Shiga toxin-producing Escherichia coli strain associated with romaine lettuce: USA, 2018–2019. Epidemiol. Infect. 150, e16. doi: 10.1017/S0950268821002703

PubMed Abstract | CrossRef Full Text | Google Scholar

Weller, D., Belias, A., Green, H., Roof, S., and Wiedmann, M. (2020). Landscape, water quality, and weather factors associated with an increased likelihood of foodborne pathogen contamination of New York streams used to source water for produce production. Front. Sustain. Food Sys. 3, 124. doi: 10.3389/fsufs.2019.00124

PubMed Abstract | CrossRef Full Text | Google Scholar

Weller, D., Kovac, J., Kent, D. J., Roof, S., Tokman, J. I., Mudrak, E., et al. (2017). Escherichia coli transfer from simulated wildlife feces to lettuce during foliar irrigation: a field study in the Northeastern United States. Food Microbiol. 68, 24–33. doi: 10.1016/j.fm.2017.06.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Weller, D., Wiedmann, M., and Strawn, L. K. (2015). Spatial and temporal factors associated with an increased prevalence of Listeria monocytogenes in spinach fields in New York state. Appl. Environ. Microbiol. 81, 6059–6069. doi: 10.1128/AEM.01286-15

PubMed Abstract | CrossRef Full Text | Google Scholar