Testing Messages on Facebook to Promote Use of an HPV Educational Web-Intervention

In the US, the human papillomavirus (HPV) vaccine remains underutilized leading to disparities in HPV-related diseases. Latinx have some of the highest rates of cancer caused by HPV. In a previous study, we developed a tailored-messaging based online educational intervention (CHICOS) that was found to increase HPV vaccination intention among Latinx participants. The current research uses Facebook Advertising to test the comparative effectiveness of messages designed using the Extended Parallel Processing Model (EPPM) to promote the use of CHICOS among Latinx young adults and parents of adolescents. We also looked at differences in the effectiveness of messages that highlighted HPV-related cancers, genital warts, or a control condition as well as differences in Spanish vs. English messages. Results found Latinx young adults and parents, were more likely to click on Facebook Advertisements containing messages in Spanish and those that mention cancer risks pertinent to this population compared to those in English or messages that discuss genital warts. Thus, findings suggest that Facebook Advertising has the potential to be a useful tool for motivating information seeking online about HPV vaccination.

Introduction

Human papillomavirus (HPV) is the most common sexually transmitted infection worldwide (1). Latinx have higher rates of HPV-related cancers compared to other groups in the United States (1). Previous research demonstrates the efficacy of health communication efforts that are customized to include culturally relevant information (2–5). The current study presents a comparison of health messages aimed at promoting HPV information seeking as a means for encouraging HPV vaccination among a Latinx audience.

The burden of HPV infection exacts a significant emotional, financial, and medical toll (6, 7). Although rates of HPV infection are similar across nearly all race and ethnicity groups, there are significant disparities in HPV-associated cancers among Latinx. For example, Latinas have the highest risk of developing cervical cancer when compared to all other US population groups—similarly, Latinos have the highest risk of developing penile cancer (1, 7–10). Additionally, Latinx experience higher rates of other HPV-related illnesses like genital warts, abnormal Pap smears, anal cancer, and oropharyngeal cancer (6).

Although a vaccine for HPV has been available and recommended by the Centers for Disease Control and Prevention (CDC) for the past 10 years for routine use among all adolescents ages 11 years and older, US vaccination rates continue to remain low (11). As of 2019, it is estimated that only 56.8% of girls and 51.8% of boys ages 13–17 nationally were up-to-date on the HPV vaccine series (11). Vaccination series completion rates are even lower among Latinx adolescents (46.2% for girls, 35.0% for boys) and significantly below the national vaccination target level of 80% coverage (12). Without significant increases in HPV vaccination, especially among high-risk populations, disparities in HPV-related cancers and other diseases are likely to continue. Thus, there is an urgent need for interventions that employ culturally-targeted strategies to reduce these disparities.

In a previous study, we developed a web-based educational intervention targeting Latinx parents of adolescents and young adults, called CHiCOS (13). CHiCOS uses responses to a series of survey items to provide personally tailored information about HPV. Results from a randomized controlled study comparing CHiCOS to “usual care” demonstrated that exposure to CHiCOS led to increased intention to vaccinate for HPV. These increased intentions unfortunately did not lead to subsequent increases in vaccination receipt. Later analysis indicated that this lack of effect was most likely due to logistical barriers to receiving the vaccine. Given that positive intentions are recognized as a necessary upstream factor to successful vaccination (14–19), the CHICOS intervention can be added to the growing list of evidence based communication strategies to improve vaccination intention.

Thus, CHiCOS presents the opportunity to further explore methods for the dissemination of evidence-based communication interventions which increase HPV vaccination intent. Previous attempts to disseminate similar web-based interventions in healthcare office waiting rooms have produced lack-luster results (13)—demonstrating the need to discover alternative methods for disseminating CHiCOS in a manner that stimulates the health information seeking behavior necessary to engage with the website. Health information seeking is particularly relevant as previous research has demonstrated its role in promoting HPV vaccine intentions (20–22). For the purposes of the current study, the primary outcome of interest is to stimulate use of the CHiCOS website—a form of health information seeking specific to HPV vaccination—via social media advertising. As such, the primary aim of the current study is to compare the effectiveness of different messages for promoting HPV information seeking via CHiCOS.

The Extended Parallel Process Model [EPPM; (23)] is a theoretical framework for designing messages that has previously proven successful for promoting HPV vaccination (24, 25). The EPPM posits that message design strategies that highlight risk severity and the target audiences' susceptibility must be coupled with messages prompting self-efficacy and response efficacy in order to effectively persuade individuals to take protective action (26). As people may underestimate their susceptibility to and the severity of HPV-related diseases (27–31)—especially cancers outside of cervical cancer including those that affect men (i.e., anal, penile, and oropharyngeal cancers)—the EPPM is a particularly relevant theoretical framework for designing messages to promote HPV vaccination. Previous research on using the EPPM to design messages related to HPV vaccination has primarily focused on intention to vaccinate as the primary outcome (32). However, other work has demonstrated the utility of the EPPM for promoting health information seeking in areas such as meningitis vaccination (33).

Results of previous studies demonstrate that message content (cervical cancer vs. genital warts) may differently influence parents' vs. young adults' intention to vaccinate [cf. (25)]. However, these messages have not been explicitly tested among a Latinx population—a group with distinct cultural norms and values related to sexual health and healthcare decision making. The current study extends this line of research. We posit the following hypotheses and research questions:

H1: Among Latinx parents of HPV-vaccination eligible adolescents, messages focusing on HPV-related cancers will lead to higher information seeking via CHICOS (i.e., link click–through rates) than messages focusing on genital warts.

H2: Among Latinx young adults, messages focusing on genital warts will lead to higher information seeking via CHICOS (i.e., link click–through rates) than messages focusing on HPV-related cancers.

RQ1: Do messages containing EPPM variables lead to higher click-through rates than messages without these variables?

RQ2: Does message language affect click-through rates?

Materials and Methods

The current study compared message effectiveness for promoting use and dissemination of CHiCOS, a web-based educational intervention [previously described in detail, see (13, 34)]. We used Facebook Advertising to test nine messages designed using the EPPM (see Figure 1) and employed a 2 × 3 × 2 mixed factorial design that compared two key decision-making audiences (Latinx young adults vs. parents of adolescents), three message topic frames (genital warts vs. cancer vs. control), and two message languages (English vs. Spanish). Per the EPPM, messages in the cancer and genital warts frames included susceptibility and severity information specific to Latinx populations, as well as response efficacy statements emphasizing the effectiveness of the HPV vaccine. Self-efficacy for seeking information about the HPV vaccine was induced by directing users to a link where more information about HPV could be found (i.e., the CHiCOS website). Messages in the control frame were written to be intentionally vague, did not mention cancer or genital warts, and did not meet criteria for containing all EPPM message components.

FIGURE 1

Figure 1. Message frames.

Facebook Advertising was chosen as the method of study for its popularity and ability to reach target audience members, the capacity to embed website links to the CHiCOS website in advertising messages, and Facebook's role in facilitating health information seeking (35, 36). Advertisement sets were created for each message frame containing three advertisements, each with a different message from the frame (see Figure 1). All advertisements used the same stock photo of multicultural youth. To comply with Facebook's requirements for establishing a source account to sponsor advertisements, we created a Facebook Page with information about the CHiCOS website. The account for this page was then used as the sponsoring source for all advertisements (see Figure 2). Facebook advertisements ran for 39 consecutive days in October and November 2017.

FIGURE 2

Figure 2. Example of facebook advertisements.

Subjects

Facebook's advertising settings provided the ability to target different population groups for the study. Messages were targeted to Latinx young adults using the Facebook Advertisement criteria: Ages 18–27, Behaviors: Multicultural Affinity: Hispanic (US - All), and geographically relevant to the study team (i.e., Denver metro area) zip codes with high Latinx populations. Similarly, Latinx parents of adolescents were targeted using the criteria: Ages 28–65 (to exclude overlap with young adult advertising sets), Behaviors: Multicultural Affinity: Hispanic (US - All), Parents, and Denver metro area zip codes with high Latinx populations. Different zip codes were used for each message frame advertising set (see Figure 1) so that users did not overlap between the six message conditions. Using population data for each zip code, we created advertising sets that were estimated to reach 18,000–19,000 unique users each.

Data Sources

Data were collected using Facebook analytics, Google analytics, and CHiCOS website paradata in order to determine the comparative effectiveness of message frames based on each advertisement's performance metrics. The primary outcome of interest was click-through rates to the CHiCOS website—wherein, for the purposes of our study, information seeking behavior was defined as clicking on a link embedded in each advertisement that led to the CHiCOS website.

Facebook Analytics

Metrics provided by Facebook included: (1) Reach - the number of unique users who saw the ad at least once, (2) Impressions – the number of total times the ad was on screen for the targeted audience, (3) Frequency - the average number of times each user saw the ad, (4) Link clicks – the number of clicks on a link within the ad that led to the CHiCOS website, and (5) Link click-through rate (CTR) - the percentage of times people saw the ad and performed a link click (i.e., link clicks/impressions), (6) Unique link clicks – the number of unique users who clicked on the website link (i.e., removes duplicate link clicks by the same user), and (7) Unique CTR – the percentage of unique users who saw the ad and performed a link click (37).

Since Facebook uses a proprietary algorithm to determine the priority by which it places advertisements in users' timelines, parameters for controlling the reach of ads were not entirely within our control. Our goal was for each ad set to reach at least 3,000 users in order to compare performance of our primary outcome (Unique CTR). However, Facebook does not allow advertisers to place limits on Reach – only on the total amount of money spent for each ad. Thus, for the purposes of this study, Reach serves as an outcome that reflects ad performance as the algorithm adjusts to extend the reach of certain ads that produce more results (e.g., higher engagement) early on and to limit the reach of ads that produce lower results. This algorithm frequently changes and Facebook does not disclose the exact parameters they use to extend or limit ad reach.

Google Analytics

Metrics provided by Google analytics (which was installed on the CHiCOS website) included: (1) Sessions - the period time a user is actively engaged with the website, (2) Pageviews – the total number of web pages viewed on the website, (3) Avg. session duration - the average time length of a session, (4) Page depth - the average number of pages viewed during a session, (5) Bounce rate – percentage of sessions with only one page view out of all sessions (38). Google analytics tracked the website referral source (Facebook) using unique referral links embedded into different advertising message frames. Thus, we were able to track the above metrics for each message frame.

CHiCOS Website Para-Data

The CHiCOS website reported metrics on the total number of time spent on the website (for users who registered to use the site by clicking the continue button on the landing page), as well as whether or not (True/False) the user completed the survey items used to tailor the educational messaging.

Analysis

Descriptive statistics as well as chi-square tests for categorical data, and z-tests and g-tests for proportions, were used to examine differences between groups (H1-2, RQ 1-2).

Results

Facebook Analytics

Overall, messages reached 73,081 Facebook users and generated 238,365 impressions for a frequency of 3.23 impressions per user. Table 1 provides a breakdown of all reach, impression, and frequency data by message frame for parents and young adults.

TABLE 1

Table 1. Facebook analytics for reach, impressions, and frequency by message frame.

A chi-square test of independence was performed to examine the relation between language and message frame. For both parents and young adults, the relationship between these variables was significant [parents: X2 (1, N = 31,975) = 171.16, p < 0.001; young adults, X2 (1, N =41,106) = 52.87, p < 0.001.). Across both parents and young adults, advertisements in Spanish reached more users than messages in English—and messages that mentioned genital warts reached more users than messages about cancer or the control condition.

Overall, Facebook advertisements produced 2,159 total link clicks and 1,552 unique link clicks to the CHiCOS website across all message conditions. The overall Unique CTR was 0.9% (per impression) and the unique CTR was 2.12% (per user reached). Table 2 provides a breakdown of link clicks and CTR by message frame for parents and young adults.

TABLE 2

Table 2. Facebook analytics for link clicks and CTR by message frame.

For Latinx parents, messages reached 31,975 users and generated 676 Unique link clicks for an average Unique CTR of 2.10%. Messages mentioning cancer had a significantly higher Unique CTR (2.49) than control (1.89) and genital warts message (1.91, p = 0.001)—thus, our hypothesis that cancer messages would produce higher CTR among Latinx parents (H1) was confirmed. Spanish messages had a higher overall Unique CTR (2.35) than English messages (p = 0.001, RQ2).

For Latinx young adults, messages reached 41,106 users and generated 876 Unique link clicks for an average Unique CTR of 2.15%. Messages mentioning cancer had a significantly higher Unique CTR (2.55) than control (1.93) and genital warts messages (1.96, p = 0.001)— disconfirming our hypothesis that genital wart messages would produce higher CTR among Latinx young adults (H2). There were no significant differences in the overall Unique CTR for English (2.14%) vs. Spanish messages (2.12%).

Google Analytics

Session metrics from Google Analytics demonstrated similar numbers for total new visitor website sessions (1,592; with confirmed referrals from Facebook) when compared to Facebook Analytics' Unique link clicks (1,552). Table 3 provides a full breakdown of Google Analytic metrics by message frame for parents and young adults. Across both user groups, the bounce rate was high, 92.58%–indicating that the majority of users did not proceed past the first page of the website. For those that did proceed past the first page of the website, the average session duration was 24.64 seconds with a page depth average of 1.21 pages per user session.

TABLE 3

Table 3. Google analytics data.

Additionally, Google's analytics showed that, in addition to the 1,592 new visitor sessions, 261 users returned the CHiCOS website at least once generating 565 additional sessions (for a total of 2,157 sessions). The bounce rate was similarly high among returning users (92.21%); however, average session duration was considerably longer at 53.01 seconds.

CHiCOS Website Para-Data

While Facebook advertising messages produced 1,552 link clicks to the website, paradata tracking new users to the website showed that only 11 people interacted with the website—and of those, only three people completed the survey items necessary to receive the personally tailored educational materials. The average time among those who interacted with the website was 8:52 minutes.

Discussion

The current study provides evidence that using Facebook advertising was an effective means for promoting information seeking about HPV vaccination via the CHiCOS website. Overall, messages in Spanish were more effective at reaching Latinx young adults and parents of adolescents than were messages in English. Among Latinx parents, messages that mentioned the relationship between HPV and cancer produced higher proportions of information seeking behavior (i.e., unique CTR) than messages that referenced genital warts or the control messages. However, among Latinx young adults, we did not find the reverse trend (as we hypothesized in H2)—that is, messages mentioning genital warts did not generate higher unique CTR than messages about cancer or the control messages.

Messages that contained all four elements as prescribed by the EPPM did not consistently outperform messages without (i.e., the control message frame). This finding provides further support for results of a previous message testing study where survey participants were shown the same messages used here containing a hyperlink to “more information” (Reno and Dempsey, unpublished manuscript). Alternatively, participants could choose to continue on in the survey without clicking the hyperlink. In this study, participants who received a control message were more likely to click the hyperlink than those who received either the cancer or genital warts messaging. Furthermore, when asked to report the extent to which a message made them “feel frightened,” participants who received the control messages consistently reported higher levels of fear than those who received the cancer or genital warts messages containing statements of severity and susceptibility designed to induce fear. Thus, further research is needed to understand the role of message design in promoting information seeking, particularly about HPV vaccination. It may be that messages that present a more veiled threat (as opposed to containing explicit susceptibility and severity statements) are more likely to inspire information seeking behavior.

We had 261 users total who visited the website more than once, demonstrating that messages disseminated via Facebook advertising were effective at stimulating initial information seeking behavior (i.e., clicking on a link for more information). However, our study did not produce evidence this information seeking behavior was carried out once users reached the CHiCOS website, as demonstrated by the high bounce rate even among repeat users—although there is some evidence that repeat users invested more time in information seeking via CHiCOS as their total time spent on the website (i.e., avg. session duration) was longer than new users. This may be due to limitations in the design of the CHiCOS website where users where first instructed to complete a short survey before receiving personalized information about the HPV vaccine. Thus, it stands to reason that users were not compelled to seek information to the extent needed to complete the survey items before receiving more information about the HPV vaccine. Further research is necessary to determine type of message content and format sufficient to engage users in HPV information seeking.

Limitations

A primary limitation to the current work is the format of the CHiCOS website. We have inferred from the data based on the large discrepancy between those who clicked on the link to the website and those who completed the survey questions necessary to view the educational information about HPV that the design of the website may be a primary barrier to additional information seeking. However, it is also probable that the messages delivered via Facebook advertising were insufficient to inspire prolonged information seeking behavior. The design of the study is limited in its ability to substantively differentiate between these two plausible explanations. Thus, further research is needed to determine the ability of Facebook advertising to promote information seeking and dissemination of web-based interventions for HPV.

Additionally, the limitations inherent to the secrecy of the Facebook algorithm impeded our ability to control for variations in advertisement reach and impressions. Thus, while this approach provided a pragmatic test of comparative message effectiveness for promoting the dissemination and use of CHiCOS, we cannot construe the extent to which variations in reach and impressions were due to aspects of the message or of the function of the algorithm. Future research should further explore the utility of Facebook Advertising for this type of A/B message testing—a feature that has more recently been added to the Facebook Advertising platform (37).

Conclusion

Overall, results suggest that Facebook Advertising is an effective means for disseminating and inspiring interest in web-based health information resources about HPV vaccination. As vaccine hesitancy continues to grow, it is essential to identify effective means to deliver accurate information about vaccines such as the HPV vaccine and other effective practices for promoting vaccination.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Colorado Multiple Institutional Review Board (COMIRB). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

JR was responsible for the overall content as guarantor. Both authors contributed to the planning, conduct, and reporting of the work described in the article.

Funding

This work was supported by internal institutional research funding.

Conflict of Interest

At the time of the study, AD was an advisory board member for Merck and Pfizer.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank Lauren Butts, Aubrey Gordon, and Kate Miller (all formerly with Sprocket Communications) for their work in facilitating the implementation of Facebook Advertising methods as well as the collection of Facebook Analytics and Google Analytics data.

References

1. Centers for Disease Control and Prevention. CDC - Human Papillomavirus (HPV) and Cancer. HPV and Cancer (2019). Available online at: https://www.cdc.gov/cancer/hpv/ (accessed December 2020).

2. Barrera M, Castro FG, Strycker LA, and Toobert DJ. Cultural adaptations of behavioral health interventions: a progress report. J Consult Clin Psychol. (2013) 81:196–205. doi: 10.1037/a0027085

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Davis RE, and Resnicow K. The cultural variance framework for tailoring health messages. In: Cho H, editor. Health Communication Message Design: Theory and Practice. Thousand Oaks, CA: SAGE Publications, Inc. (2011). p. 115–36.

Google Scholar

4. Kreuter MW, Lukwago SN, Bucholtz DC, Clark EM, and Sanders-Thompson V. Achieving cultural appropriateness in health promotion programs: targeted and tailored approaches. Health Educ Behav. (2003) 30:133–46. doi: 10.1177/1090198102251021

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kreuter MW, and McClure SM. The role of culture in health communication. Annu Rev Public Health. (2004) 25:439–55. doi: 10.1146/annurev.publhealth.25.101802.123000

CrossRef Full Text | Google Scholar

6. American Cancer Society. Cancer Facts & Figures| Hispanics-Latinos. American Cancer Society. American Cancer Society (2018). Available online at: https://www.cancer.org/research/cancer-facts-statistics/hispanics-latinos-facts-figures.html (accessed December 2020).

Google Scholar

7. Jemal A, Ward EM, Johnson CJ, Cronin KA, Ma J, Ryerson AB, et al. Annual report to the nation on the status of cancer, 1975–2014, featuring survival. J Natl Cancer Inst. (2017) 109:djx030. doi: 10.1093/jnci/djx030

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Ortiz AP, Engels EA, Nogueras-González GM, Colón-López V, Soto-Salgado M, Vargas A, et al. Disparities in human papillomavirus–related cancer incidence and survival among human immunodeficiency virus–infected Hispanics living in the United States. Cancer. (2018) 124:4520–8. doi: 10.1002/cncr.31702

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Sharma P, Zargar-Shoshtari K, Pettaway CA, Schabath MB, Giuliano AR, and Spiess PE. Disparities in penile cancer. Cancer Control.. (2016) 23:409–14. doi: 10.1177/107327481602300412

CrossRef Full Text | Google Scholar

10. Center for Disease Control and Prevention. United States Cancer Statistics, Data Visualizations. USCS Data Visualizations. CDC (2016). Available online at: https://gis.cdc.gov/grasp/USCS/DataViz.html (accessed December 2020).

11. Elam-Evans LD. National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 years—United States, 2019. MMWR Morbid Mortal Wkly Rep. (2020) 69:1109–16. doi: 10.15585/mmwr.mm6933a1

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Reagan-Steiner S. National, regional, state, and selected local area vaccination coverage among adolescents aged 13–17 years—United States, 2015. MMWR Morbid Mortal Wkly Rep. (2016) 65:1109–16. doi: 10.15585/mmwr.mm6533a4

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Dempsey AF, Maertens J, Sevick C, Jimenez-Zambrano A, and Juarez-Colunga E. A randomized, controlled, pragmatic trial of an iPad-based, tailored messaging intervention to increase human papillomavirus vaccination among Latinos. Hum Vaccines Immunother. (2019) 2019:1–8. doi: 10.1080/21645515.2018.1559685

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Askelson NM, Campo S, Lowe JB, Smith S, Dennis LK, and Andsager J. Using the theory of planned behavior to predict mothers' intentions to vaccinate their daughters against HPV. J School Nurs. (2010) 26:194–202. doi: 10.1177/1059840510366022

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Askelson NM, Campo S, Smith S, Lowe JB, Dennis LK, and Andsager J. The birds, the bees, and the HPVs: What drives mothers' intentions to use the HPV vaccination as a chance to talk about sex? J Pediatr Health Care. (2011) 25:162–70. doi: 10.1016/j.pedhc.2010.01.001

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Catalano HP, Knowlden AP, Birch DA, Leeper JD, Paschal AM, and Usdan SL. Using the Theory of Planned Behavior to predict HPV vaccination intentions of college men. J Am Coll Health. (2017) 65:197–207. doi: 10.1080/07448481.2016.1269771

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Gerend MA, and Shepherd JE. Predicting human papillomavirus vaccine uptake in young adult women: comparing the health belief model and theory of planned behavior. Ann Behav Med. (2012) 44:171–80. doi: 10.1007/s12160-012-9366-5

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Juraskova I, O'Brien M, Mullan B, Bari R, Laidsaar-Powell R, and McCaffery K. HPV vaccination and the effect of information framing on intentions and behaviour: an application of the theory of planned behaviour and moral norm. Int J Behav Med. (2012) 19:518–25. doi: 10.1007/s12529-011-9182-5

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Park H-M, and Oh H-E. Factors associated with the intention of human papillomavirus vaccination among mothers of junior high school daughters. J Korea Contents Assoc. (2014) 14:307–18. doi: 10.5392/JKCA.2014.14.08.307

CrossRef Full Text | Google Scholar

20. Baldwin AS, Bruce CM, and Tiro JA. Understanding how mothers of adolescent girls obtain information about the human papillomavirus vaccine: associations between mothers' health beliefs, information seeking, and vaccination intentions in an ethnically diverse sample. J Health Psychol. (2013) 18:926–38. doi: 10.1177/1359105312445078

PubMed Abstract | CrossRef Full Text | Google Scholar

21. McRee A-L, Reiter PL, and Brewer NT. Parents' Internet use for information about HPV vaccine. Vaccine. (2012) 30:3757–62. doi: 10.1016/j.vaccine.2011.11.113

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Wong NCH. Predictors of information seeking about the HPV vaccine from parents and doctors among young college women. Commun Q. (2014) 62:75–96. doi: 10.1080/01463373.2013.860905

CrossRef Full Text | Google Scholar

23. Witte K. Fear control and danger control: a test of the extended parallel process model (EPPM). Commun Monogr. (1994) 61:113–34. doi: 10.1080/03637759409376328

CrossRef Full Text | Google Scholar

24. Carcioppolo N, Jensen JD, Wilson SR, Collins WB, Carrion M, and Linnemeier G. Examining HPV threat-to-efficacy ratios in the Extended Parallel Process Model. Health Commun. (2013) 28:20–8. doi: 10.1080/10410236.2012.719478

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Krieger JL, and Sarge MA. A serial mediation model of message framing on intentions to receive the human papillomavirus (HPV) vaccine: revisiting the role of threat and efficacy perceptions. Health Commun. (2013) 28:5–19. doi: 10.1080/10410236.2012.734914

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Witte K, Meyer G, and Martell D. Effective Health Risk Messages: A Step-By-Step Guide. Thousand Oaks, CA: Sage (2001).

Google Scholar

27. Garcés-Palacio IC, and Scarinci IC. Factors associated with perceived susceptibility to cervical cancer among Latina immigrants in Alabama. Matern Child Health J. (2012) 16:242–8. doi: 10.1007/s10995-010-0737-x

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Lau JT, Wang Z, Lau M, and Lai CH. Perceptions of HPV, genital warts, and penile/anal cancer and high-risk sexual behaviors among men who have sex with men in Hong Kong. Arch Sex Behav. (2014) 43:789–800. doi: 10.1007/s10508-013-0172-3

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Nan X, Zhao X, and Briones R. Parental cancer beliefs and trust in health information from medical authorities as predictors of HPV vaccine acceptability. J Health Commun. (2014) 19:100–14. doi: 10.1080/10810730.2013.811319

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Oh J-K, Lim MK, Yun EH, Lee E-H, and Shin H-R. Awareness of and attitude towards human papillomavirus infection and vaccination for cervical cancer prevention among adult males and females in Korea: a nationwide interview survey. Vaccine. (2010) 28:1854–60. doi: 10.1016/j.vaccine.2009.11.079

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Wheldon CW, Daley EM, Buhi ER, Nyitray AG, and Giuliano AR. Health beliefs and attitudes associated with HPV vaccine intention among young gay and bisexual men in the southeastern United States. Vaccine. (2011) 29:8060–5. doi: 10.1016/j.vaccine.2011.08.045

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Fu LY, Bonhomme L-A, Cooper SC, Joseph JG, and Zimet GD. Educational interventions to increase HPV vaccination acceptance: a systematic review. Vaccine. (2014) 32:1901–20. doi: 10.1016/j.vaccine.2014.01.091

PubMed Abstract | CrossRef Full Text | Google Scholar

33. So J, Kuang K, and Cho H. Information seeking upon exposure to risk messages: predictors, outcomes, and mediating roles of health information seeking. Commun Res. (2019) 46:663–87. doi: 10.1177/0093650216679536

CrossRef Full Text | Google Scholar

34. Maertens JA, Jimenez-Zambrano AM, Albright K, and Dempsey AF. Using community engagement to develop a web-based intervention for Latinos about the HPV vaccine. J Health Commun. (2017) 22:285–93. doi: 10.1080/10810730.2016.1275890

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Volkman JE, Hokeness KL, Morse CR, Viens A, and Dickie A. Information source's influence on vaccine perceptions: an exploration into perceptions of knowledge, risk and safety. J Commun Healthc. (2020). doi: 10.1080/17538068.2020.1793288

CrossRef Full Text | Google Scholar

36. Westbrook L, and Zhang Y. Questioning strangers about critical medical decisions: what happens if you have sex between the HPV shots? Information Res. (2015) 20:667.

Google Scholar

37. Facebook for Business. A/B Tests Types Available on Facebook. Facebook Business Help Center (2020). Available online at: https://www.facebook.com/business/help/1159714227408868 (accessed December 2020).

38. Google. Dimensions and Metrics - Analytics Help. Analytics Help (2020). Available online at: https://support.google.com/analytics/answer/1033861?hl=en (Accessed December 29, 2020).