Recruiting farm parents for a 6-week online child safety survey study using paid Facebook advertisements

Agricultural populations are a popular target for research due to the industry's dangerous conditions, unique work environments, and youth-including workforce. However, reaching and recruiting eligible participants is challenging and expensive when compared to other industries. We conducted a two-phase, multi-week paid advertising campaign on Facebook.com, varying delivery time, imagery, verbiage, and targeting methods to recruit U.S. farm parents for an online survey study investigating childhood agricultural safety. Advertisements were active for 4 weeks in fall 2021 and 3 weeks in winter 2022 at $1,500 per week. The fall recruitment targeted farm parents, depicting three farm-related images, while the winter recruitment targeted all parents, depicting farm-rescue trainings with firefighters. The fall recruitment garnered 5,535 link clicks with a reach of 233,690 ($1.07 per click). The winter recruitment garnered 8,602 clicks with a reach of 750,764 ($0.53 per click) and higher user engagement. A total of 1,439 participants began the screener questionnaire, a conversion rate of 10.18%. Of 815 completed responses, 271 met our inclusion criteria. One hundred and sixty-four participants completed the study: 45 from fall (27.6% dropout) and 119 (40% dropout) from winter. The overall attrition rate was 38.1% and cost per completed response was $64 USD. We successfully recruited our target sample size for this study. Notably, advertisement timing, imagery, and sampling frame likely affected performance. A screening questionnaire was imperative in identifying sham responses. These findings show that paid Facebook advertising can be a feasible recruitment tool to engage with a traditionally difficult to reach population with proper precautions and planning.

1. Introduction

Agricultural populations are a popular target for research due to the industry's dangerous conditions, unique work environments, and youth-including workforce (Nilsson, 2016; US Bureau of Labor Statistics, 2021; National Children's Center for Rural Agricultural Health Safety, 2022). However, reaching and recruiting eligible participants is often challenging, time-consuming, and expensive when compared to other industries and populations (Glas et al., 2019; Turland and Slade, 2020; Weigel et al., 2021). Traditionally, contact information would be acquired or purchased from data vendors, extension agencies, or government agencies, and then participants randomly selected, or random digit dialing is conducted for broader scopes. However, these can often be expensive and introduce unique biases as cultural changes affect traditional recruitment methods (Couper, 2017). For example, Pew Research reports a 27% point decrease in phone telephone response rates from 1997 to 2016 (Keeter et al., 2017). Similarly, address-based sampling, often acquired through the US Postal Service's Delivery Sequence File, can provide a large number of potential participants with high accuracy, but can require time and labor intensive eligibility verification (Iannacchione et al., 2003).

Historically, phone and physical address-based recruitment methods have been used and evaluated for agricultural populations (Glas et al., 2019; Weigel et al., 2021). However, a notable challenge for these populations is identifying eligible participants from a larger sample. Several private and public agencies exist which house these specific datasets, but this introduces additional cost, time, and regulatory oversight. As such, in order to account for changing population habits and recruitment efficacies, social media (e.g., Facebook and Twitter) has been increasingly used to recruit large and diverse groups, demonstrating it's potential to be a useful and cost-effective recruitment tool for research (Ramo and Prochaska, 2012; Frandsen et al., 2016; Jones and Salazar, 2016; Lee et al., 2020).

Home to both Facebook and Instagram, parent company Meta is the largest social media company operating today. Facebook.com reports over 1.8 billion active users-−200 million within the US—and is trending upwards as more Americans use the internet (Pew Research Center Internet Technology, 2019; Facebook, 2022). Facebook provides tools for targeting unique and traditionally hard to reach populations based on data provided by individuals and gathered by monitoring users' activities on the platform. The integrated ads manager platform within Facebook.com allows for targeting based on interests, industry, demographics, etc. which help alleviate the labor intensive process of creating the sampling frame. While the accuracy of this labeling has been called into question (Jones et al., 2012), this remains a popular tool for improving advertisement accuracy and cost efficiency (Schneider and Harknett, 2022).

Despite this, there is little evidence regarding social media's effectiveness in recruiting agricultural populations. As rural areas of the US gain internet access, it is likely online methodologies for recruiting and researching these populations also increase, as seen in telemedicine and remote work opportunities exacerbated by the COVID-19 pandemic (Perrin, 2019; Hirko et al., 2020; Rush et al., 2021).

This study employed an expansion of duration and budget, from a past pilot test through Facebook.com paid advertisements (Burke et al., 2021). In this paper, we describe our two-phase, multi-week paid advertising campaign on Facebook.com, varying delivery time, imagery, verbiage, and targeting methods to recruit U.S. farm parents for an online survey study investigating childhood agricultural safety.

2. Methods

We ran a two-phase paid advertisement campaign on Facebook.com, via the Meta Business Suite, from Sep 27, 2021 to October 21, 2021 and from Jan 27, 2022 to Feb 17, 2022, hosted on the Facebook page for the National Children's Center for Rural and Agricultural Health and Safety (NCCRAHS). The first, fall phase was targeted toward parents aged 18 and above with employment in the “Farming, Fishing, and Forestry” industry as defined by Facebook as “People with roles in farming, fishing and forestry. Examples include: farmer, rider, crew member, handyman, etc.” with an estimated user population of 2,000,000. The second, winter phase was targeted toward all parents nationwide, farm and otherwise, aged 18 and above as defined by Facebook with an estimated population of 200,000,000. Both phases were targeted to individuals within the US.

The fall phase included three images: a child feeding a calf with a bottle, a girl sweeping a barn in full personal protective equipment, and an overturned tractor (Figures 1A–C), advertised concurrently. The winter phase instead included two images of a night-time firefighter training, one featuring an overturned tractor for weeks 1 and 2 and one featuring a grain bin for weeks 3 and 4 (Figures 2A, B, respectively; week 4 was not conducted as the target recruitment number was reached). Both phases included verbiage such as “Farm parents wanted!” and “See if you're eligible for on online, paid survey study.” Both phases were budgeted at $6,000 USD each charged per-click.

FIGURE 1

Figure 1. Fall campaign advertisements depicting a girl sweeping in full personal protective equipment (A), a child feeding a calf (B), and an overturned tractor (C).

FIGURE 2

Figure 2. Winter campaign advertisements depicting a firefighter training of an overturned tractor (A) and a grain bin rescue (B).

All advertisement clicks led to a REDCap-hosted consent form followed by a brief 10-item screener questionnaire asking participants about farm and family demographics, alongside a description of the full research study. Participants were compensated $50 USD for full completion of the study, informed during consent. Participants were then invited to the full study on a weekly rolling basis pending manual review of screener response integrity, such as identifying non-sensical farm demographic responses (e.g., reported a beef operation with no cattle). This study consisted of a pre- and post-test survey measuring knowledge, attitudes, and intended behaviors toward childhood agricultural safety. Experimental and control group participants were randomly assigned to read through 4 weekly news articles on a childhood agricultural injury incident or non-health and safety-focused farm life articles, respectively. The reading of articles occurred between the pre- and post-tests.

All advertisements were closely monitored by our research team for community engagement and public comment. Derogatory comments were hidden (visible only to the poster and their Facebook friends). Meanwhile, public questions and comments were responded to by our team, through the NCCRAHS Page, where appropriate. The winter phase was terminated after 3 weeks and $4,627 spent as we reached our target study enrollment of 200 participants and at least 160 completed responses. Throughout the campaign, ad performance was monitored for engagement, reach, impressions, clicks, click-through-rate, survey throughput, and attrition rates.

3. Results

3.1. Advertisement performance

The fall phase was completed at a total cost of $5,944 and the winter phase at $4,627. The fall phase saw 23 shares, 137 reactions, and 27 comments with most being neutral or an inquiry for more information (Table 1). This phase saw 5,535 clicks leading to 113 partial screener completions and 243 full completions (Figure 3). Of these, 76 were determined to be eligible for the full study and were invited, 45 of whom completed the full survey study. This yielded a cost per click of $1.07 with a cost per completed response of $132.

TABLE 1

Table 1. Advertisement performance analytics by campaign, week, and image.

FIGURE 3

Figure 3. Comparison of fall and winter campaign metrics.

Images of the girl sweeping and the child feeding the calf (Figures 1A, B, respectively) performed similarly, however the tractor rollover image (Figure 1C) yielded much better results with nearly double the combined reach, impressions, and clicks of the other two images. Notably, due to the tractor image's success, we reallocated all designated advertisement funding to only this image for week four. Prior to week four, the tractor image had similar cost per click to the other images.

The winter phase garnered much greater engagement and performance metrics with 91 shares, 196 reactions, and 52 comments (Table 1). This phase saw 8,719 clicks leading to 221 partial screener completions and 574 full completions (Figure 3). Of these, 195 were determined to be eligible for the full study and invited, 119 of whom completed the full survey study. This yielded a cost per click of $0.53 and a cost per completed response of $36, in line with other research studies. However, this phase saw a greater number of derogatory and inflammatory comments with three requiring immediate moderation, compared to zero in the fall phase. We also observed a more in-depth discussion among commenters, sharing personal stories of farm injuries and firefighters responding to calls on farms, including an 18 comment-long dialogue.

The two winter phase images performed similarly. While the tractor image garnered greater engagement with commenters, both had similar levels of reach, impressions, and clicks.

Overall, this 7-week advertising campaign saw a reach of 1,050,348, with 1,938,026 impressions, and 14,254 clicks at $0.74 per click; we had 79 comments, 114 shares, and 332 reactions. Not accounting for personnel time, our advertisement yielded a cost per participant of $36 and a cost per completed response of $64. We did not see any geographic biases in respondents' locations.

3.2. Post-advertisement behavior

With 14,254 link clicks and 817 completed screener questionnaires, we observed a conversion rate of 5.7%. We invited 271 eligible participants to the full study, 265 of whom consented to the pre-test questionnaire. Of these, 164 completed the full study, yielding an overall attrition rate of 38.1% and a dropout rate of 39.5%, greater than our predicted 30%. The fall campaign showed a lower dropout rate than the winter campaign at 27.6 and 40%, respectively, though with markedly fewer participants.

Notably, during the winter campaign, we observed 106 responses to the screener questionnaire completed within 90 min, compared to the average of 4.9 h between all other screener completions. Further investigation found these to be mostly non-sense responses, including participants who reported dairy operations with no animals or equipment present. These responses were deemed fraudulent and ineligible for full study enrollment.

4. Discussion

This study has demonstrated that paid advertisements on Facebook.com can be an option for recruiting a farming population for online survey studies or similar long-term surveys. We were successful in recruiting a sufficient number of participants for a 6-week online survey study, however the cost was generally high at $36 USD per eligible participant and $64 per completed response. We also observed relatively high attrition rates of 38% throughout the study. However, the costs and dropout rates can be managed with proper preparation and forethought such as enticing but reasonable incentives, personal attachment/emotional components to study messaging, and balancing study safeguards with ease of participation.

4.1. Performance differences

We observed a notable difference in performance and efficiency between our two advertising phases. Our winter phase saw nearly triple the eligible participants at nearly 1/3 the cost per completed response and half the cost per click. The three main differences we believe contributed to the success of the winter campaign over the fall are the timing, imagery, and sampling frame. The first campaign was conducted during fall—harvest and back-to-school season—while the second took place during the typically slower winter season. The additional free time of farm parents allows for greater time spent on social media and greater willingness to participate in surveys.

We also leveraged the performance of the initial ad campaign to select more eye-catching pictures for the winter phase. The first phase was split into three categories based on a safety survey: a high safety adoption image (Figure 1A, girl sweeping in full protective equipment), a neutral image (Figure 1B, child feeding a calf) and a low safety adoption image (Figure 1C, overturned tractor). Given the success of the tractor image, we selected a similar image with the focus on firefighters due to their close ties to the farming population and their role in public health and safety (Figure 2). The wording on all images remained similar throughout the study.

Lastly, the major difference in the sampling frame chosen (2,000,000 parents employed in Farming, Fishing, and Forestry vs. 200,000,000 parents in total) may have impacted performance. The frequency of ads shown to an individual person in the fall campaign was 3.6 vs. 1.55 in the winter campaign. The greater number of impressions and individuals who viewed the winter campaign likely gave a greater chance for willing and eligible participants to respond, as noted by the higher click rate though lower proportion of eligible participants among conversions. While we observed a lower portion of eligible completions of our screener tool, the winter campaign still yielded more cost-effective recruitment. These changes were made in order to increase recruitment first and foremost, making a direct comparison between campaigns difficult and inaccurate. Yet, these findings and experiences lend themselves to inform and direct future recruitment of agricultural populations.

4.2. Spam responses

Given the broad reach and mass-viewing that paid advertisements can garner, a filter between the advertised survey and the public was imperative. The mention of a paid survey opportunity is a double-edged sword: it can catalyze scammers and those looking to take advantage of the survey but it also encourages the target population to inquire, incentivizing research participation. Creating a stop-gap in participant workflow, that is, a filter for manual review of response integrity, allowed us to identify and remove over 100 fraudulent responses saving time and effort of doing so after participants have completed the full study and received incentives. However, this likely reduced our survey's conversion rate and recruitment rate as participants had an additional barrier between them and the survey study.

4.3. Comment moderation and engagement

Public advertisements with open forum community engagement yielded another avenue for benefits and drawbacks. A higher number of comments and shares can increase reach and impressions on an advertisement, showing the study to additional prospective participants without an increase in advertising costs. Presumably, these comments led to an organic spread of the advertisement into the commenters' networks, displaying on the news feeds of their network connections. However, the openness of the ads allows for bad-faith actors and inflammatory comments to affect the legitimacy of the ad, the hosting Facebook page, and any affiliated organizations.

We observed multiple comments questioning the veracity of the study, and some denouncing agricultural health and safety as regulatory overreach and child endangerment. While Facebook provides the option to hide or delete these comments, this requires a project team to be moderating the ad outside normal working hours, and the attempted controls could lead to further inflammatory responses when a comment is censored. The necessary staffing is an important consideration for research teams contemplating taking on this type of recruitment strategy. The sampling frame is also an important aspect, as the narrower targeting of just AFF occupations in the fall campaign resulted in no publicly posted comments requiring our team's immediate action. We recommend careful consideration, pre-planning, and maintaining an open dialogue with the host Page's administrative/communications team prior to initiating paid advertisement recruitment on a social media platform.

Contrary to this, increased engagement can also provide a benefit to the affiliated pages and organizations. We observed an 18-comment dialogue between farmers, firefighters, and our staff members promoting agricultural health and safety. We also saw a slight increase to the NCCRAHS Facebook Page performance analytics, and new followers of the page.

5. Conclusions

This study has shown that using paid Facebook advertisements to recruit farm parents for a long-term, online survey study is feasible with the proper precautions in place. These advertisements can introduce various uncertainties, biases, and costs not experienced with traditional recruitment methods. Screener forms, moderation teams, and careful wording/image selection can help alleviate these drawbacks in order to reach a historically difficult to recruit population.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Marshfield Clinic Research Institute Institutional Review Board. The patients/participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any identifiable images or data included in this article.

Author contributions

RB: study design, survey creation and management, Facebook advertisement creation, management and data collection, data preparation and analysis, and manuscript preparation. BW: study design, survey creation, Facebook advertisement creation and management, data analyses and interpretation, and manuscript preparation. KN: study design, survey creation, data analyses and interpretation, and manuscript preparation. MP: advisement of study design, Facebook advertisement creation and management, and manuscript preparation. All authors contributed to the article and approved the submitted version.

Funding

Funding support was provided through the National Farm Medicine Center, Marshfield Clinic Research Institute, and National Children's Center for Rural and Agricultural Health and Safety through the National Institute for Occupational Safety and Health Cooperative Agreement U54 OH009568.

Acknowledgments

The authors would like to thank Scott Heiberger for his assistance moderating the Facebook advertisements and reviewing the manuscript. The authors also thank Marie Fleisner for assistance with editing and formatting the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Burke, R. R., Weichelt, B. P., and Namkoong, K. (2021). Facebook ads manager as a recruitment tool for a health and safety survey of farm mothers: pilot study. JMIR Format. Res. 5, e19022. doi: 10.2196/19022

PubMed Abstract | CrossRef Full Text | Google Scholar

Couper, M. P. (2017). New developments in survey data collection. Ann. Rev. Sociol. 43, 121–145. doi: 10.1146/annurev-soc-060116-053613

CrossRef Full Text | Google Scholar

Facebook (2022). Insights to Go. Available online at: https://www.facebook.com/business/insights/i2g (accessed August 2, 2022).

Google Scholar

Frandsen, M., Thow, M., and Ferguson, S. G. (2016). The effectiveness of social media (facebook) compared with more traditional advertising methods for recruiting eligible participants to health research studies: a randomized, controlled clinical trial. JMIR Res. Protoc. 5, e161. doi: 10.2196/resprot.5747

PubMed Abstract | CrossRef Full Text | Google Scholar

Glas, Z. E., Getson, J. M., Gao, Y., Singh, A. S., Eanes, F. R., Esman, L. A., et al. (2019). Effect of monetary incentives on mail survey response rates for midwestern farmers. Soc. Nat. Resour. 32, 229–237. doi: 10.1080/08941920.2018.1530815

CrossRef Full Text | Google Scholar

Hirko, K. A., Kerver, J. M., Ford, S., Szafranski, C., Beckett, J., Kitchen, C., et al. (2020). Telehealth in response to the COVID-19 pandemic: implications for rural health disparities. J. Am. Med. Informat. Assoc. 27, 1816–1818. doi: 10.1093/jamia/ocaa156

PubMed Abstract | CrossRef Full Text | Google Scholar

Iannacchione, V. G., Staab, J. M., and Redden, D. T. (2003). Evaluating the use of residential mailing addresses in a metropolitan household survey. Public Opin. Quart. 67, 202–210. doi: 10.1086/374398

CrossRef Full Text | Google Scholar

Jones, J., and Salazar, L. F. (2016). A review of HIV prevention studies that use social networking sites: implications for recruitment, health promotion campaigns, and efficacy trials. AIDS Behav. 20, 2772–2781. doi: 10.1007/s10461-016-1342-9

PubMed Abstract | CrossRef Full Text | Google Scholar

Jones, R. B., Goldsmith, L., Williams, C. J., and Boulos, M. N. K. (2012). Accuracy of geographically targeted internet advertisements on google adwords for recruitment in a randomized trial. J. Medical Internet Res. 16, e146. doi: 10.2196/jmir.1991

PubMed Abstract | CrossRef Full Text | Google Scholar

Keeter, S., Hatley, N., Kennedy, C., and Lau, A. (2017). What Low Response Rates Mean for Telephone Surveys. Washington, DC: Pew Research Center.

Google Scholar

Lee, S., Torok, M., Shand, F., Chen, N., McGillivray, L., Burnett, A., et al. (2020). Performance, cost-effectiveness, and representativeness of facebook recruitment to suicide prevention research: online survey study. JMIR Mental Health 7, e18762. doi: 10.2196/18762

PubMed Abstract | CrossRef Full Text | Google Scholar

National Children's Center for Rural Agricultural Health Safety (2022). 2022 Fact Sheet—Childhood Agricultural Injuries. Marshfield, WI: Marshfield Clinic Health System. Available online at: https://marshfieldresearch.org/Media/Default/NFMC/National%20Childrens%20Center/2022_Child_Ag_Injury_Fact_Sheet_with_References.pdf (accessed August, 2, 2022).

Google Scholar

Nilsson, K. (2016). Parents' attitudes to risk and injury to children and young people on farms. PLoS ONE 11, e0158368. doi: 10.1371/journal.pone.0158368

PubMed Abstract | CrossRef Full Text | Google Scholar

Perrin, A. (2019). Digital Gap Between Rural and Nonrural America Persists. Available online at: https://www.pewresearch.org/fact-tank/2021/08/19/some-digital-divides-persist-between-rural-urban-and-suburban-america/ (accessed August 23, 2022).

Google Scholar

Pew Research Center Internet Technology (2019). Demographics of Internet and Home Broadband Usage in the United States. Available online at: https://www.pewresearch.org/internet/fact-sheet/internet-broadband/ (accessed August 23, 2022).

Google Scholar

Ramo, D. E., and Prochaska, J. J. (2012). Broad reach and targeted recruitment using facebook for an online survey of young adult substance use. J. Med. Internet Res. 14, e28. doi: 10.2196/jmir.1878

PubMed Abstract | CrossRef Full Text | Google Scholar

Rush, K. L., Seaton, C., Li, E., Oelke, N. D., and Pesut, B. (2021). Rural use of health service and telemedicine during COVID-19: the role of access and eHealth literacy. Health Informat. J. 27, 14604582211020064. doi: 10.1177/14604582211020064

PubMed Abstract | CrossRef Full Text | Google Scholar

Schneider, D., and Harknett, K. (2022). What's to like? Facebook as a tool for survey data collection. Sociol. Methods Res. 51, 49124119882477. doi: 10.1177/0049124119882477

PubMed Abstract | CrossRef Full Text | Google Scholar

Turland, M., and Slade, P. (2020). Farmers' willingness to participate in a big data platform. Agribusiness 36, 20–36. doi: 10.1002/agr.21627

CrossRef Full Text | Google Scholar

US Bureau of Labor Statistics (2021). National Census of Fatal Occupational Injuries in 2020. Washington DC: U.S. Bureau of Labor Statistics.

Google Scholar

Weigel, C., Paul, L. A., Ferraro, P. J., and Messer, K. D. (2021). Challenges in recruiting U.S. farmers for policy-relevant economic field experiments. Appl. Econ. Perspect. Policy. 43, 556–572. doi: 10.1002/aepp.13066

CrossRef Full Text | Google Scholar