Strategies to develop an LGBTQIA+-inclusive adolescent sexual health program evaluation

Introduction: Adolescent sexual health interventions are increasingly incorporating content that is inclusive of LGBTQIA+ youth (lesbian, gay, bisexual, transgender, queer/questioning, intersex, asexual, and other marginalized sexualities and genders). Evaluations of such programs must also be inclusive to enhance the validity of evaluation results and avoid further marginalization. We present strategies for increasing LGBTQIA+-inclusivity based on our evaluation of SafeSpace, a sexual health curriculum.

Methods: To design an LGBTQIA+-inclusive program evaluation, we leveraged LGBTQIA+ research staff’s insights, pursued a parental consent waiver, developed an inclusive recruitment plan, and crafted demographic and sexual behavior survey measures with input from youth and equity experts. We conducted a pilot study with 42 youth ages 14–17 to assess the feasibility and efficacy of our strategies.

Results: We obtained a parental consent waiver and recruited a majority LGBTQIA+ pilot study sample (62%). Using themes from cognitive interviews with youth and experts regarding inclusive framing and use of plain language, we refined demographic measures and expanded sexual behavior measures.

Conclusion: Findings suggest that the strategies used to enhance LGBTQIA+-inclusivity in our evaluation of SafeSpace were effective in respectfully and more accurately capturing a fuller range of experiences and identities of LGBTQIA+ and cis-straight youth. The strategies and survey measures developed for this study can be applied to increase LGBTQIA+-inclusivity in other adolescent sexual health program evaluations.

1 Introduction

Comprehensive, evidence-based sexuality education¹ has been shown to improve sexual and reproductive health (SRH) knowledge, behaviors, and outcomes for youth (1–4). However, there is growing recognition that sexuality education must be inclusive of youth who are lesbian, gay, bisexual, transgender, queer/questioning, intersex, asexual, and other marginalized sexualities and genders (LGBTQIA+²) (5), community-centered (6), and tailored to meet youth’s needs. An increasing percentage of U.S. youth identify as LGBTQIA+, representing one in four high school students in 2021 (7). And yet, the 2019 National School Climate Survey reports that only 8.2 percent of students had ever received LGBTQ +-inclusive sex education in school (8). Often, school-based sexual health curricula focus predominantly on the needs of heterosexual cisgender youth without discussion of sexual orientation and gender identity development. The curricula often exclude LGBTQIA+-inclusive examples of healthy relationships and consent. Additionally, programs typically emphasize penile-vaginal (PV) sex (9) and its associated pregnancy risk, while excluding protective practices related to other types of sexual activity particularly relevant to LGBTQIA+ youth.

There are significant consequences of inadequate sexuality education for LGBTQIA+ youth. Heteronormative and cisnormative sex education programs can alienate LGBTQIA+ youth and contribute to feelings of sexual shame and other negative mental health outcomes (10–13). Receiving sex education that is not inclusive is correlated with increased likelihood of experiencing sexual violence and engaging in sexual behaviors that increase risk for unintended pregnancy and sexually transmitted infections (STIs) (10, 12, 14). In fact, LGBTQIA+ young people assigned female at birth (AFAB) are more likely to experience unintended pregnancy than their heterosexual cisgender peers (15–18). Although less studied, evidence also suggests that trans masculine young people are at least as likely to become pregnant as cisgender young people (19). In addition to a lack of inclusive sex education, these disproportionate outcomes may be associated with experiences of stigma and discrimination (20, 21), lack of support (22), limited connectedness with family and school (23–25), and trauma from sexual assault (26).

Adapting existing interventions and developing new interventions can help address gaps in sexuality education for LGBTQIA+ youth. New digital interventions, including web- and text-based programs, have shown promise in improving identity self-acceptance, sexual health knowledge, communication skills, and contraceptive use during PV sex among LGBTQIA+ youth (27, 28). Studies describing these programs and their impacts provide valuable information to inform additional curricula that serve LGBTQIA+ youth.

Despite a growing number of programs designed to meet the sexual health needs of LGBTQIA+ youth, there is limited literature documenting how to tailor such evaluations. Standard evaluation practices may limit the ability to document impacts for LGBTQIA+ youth and even contribute to the marginalization of this population. To address this issue, we describe an LGBTQIA+-inclusive approach to evaluating SafeSpace, a mobile app-based sexuality education curriculum that was intentionally designed to resonate with LGBTQIA+ youth. We highlight specific strategies for centering the needs and experiences of LGBTQIA+ youth in all aspects of the program evaluation—from navigating Institutional Review Board (IRB) and funder requirements to recruitment and enrollment to assessing outcomes. These strategies can be broadly applied to tailor evaluations and strengthen the evidence base for programs that support the sexual health and well-being of LGBTQIA+ youth.

SafeSpace is a self-paced, mobile app-based 10-week program for youth 14 to 18 years old, assigned female or intersex at birth.³ The intervention was adapted from Real Talk, a mobile app publicly available in the Apple App Store that uses storytelling and technology to improve youth mental, emotional, and behavioral health (29). The SafeSpace program centers youths’ desire for stories from different youth perspectives and provides a sense of privacy (30). SafeSpace takes an LGBTQIA+-inclusive approach to all topics covered, including healthy relationships; identity and development; pregnancy and STI/HIV prevention; safety, communication and decision making; and accessing healthcare.

We are evaluating SafeSpace using a randomized control trial (RCT) design, which will allow us to measure attitudes, intentions, self-efficacy, and behaviors of participants who have access to SafeSpace compared to participants in the control condition, who have access to a general health app that does not include SRH information. Because 45 percent of youth who use Real Talk identify as LGBTQIA+, we are tailoring our recruitment efforts to ensure that at least half the sample identifies as LGBTQIA+ .⁴ We are also prioritizing youth of color and youth living in rural areas in the southeastern United States (31, 32).

2 Methods

2.1 Pilot implementation

From December 2022 to January 2023, we used paid social media ads to recruit 42 pilot study participants. Potential participants were invited to download SafeSpace from the Apple App Store and complete an eligibility screener within the app. Using an RCT design, eligible youth then completed a consent/assent form and baseline survey and were randomized to receive the intervention (SafeSpace SRH) or control (SafeSpace general health) app. Participants were randomized based on sexual orientation (LGBTQIA+, cisgender-straight), race and ethnicity (Hispanic regardless of race; non-Hispanic Black; and all other non-Hispanic non-Black races, such as White, Asian, Native Hawaiian or Pacific Islander, Indigenous American)⁵, and age (14–16, 17–18 years). We used Stata Version 16.1 to produce overall baseline prevalence estimates of demographic characteristics and sexual behaviors and test for differences in the distributions of sexual health behaviors by LGBTQIA+ status using chi-square tests. We focus on measures of sexual behavior particularly relevant to LGBTQIA+-inclusivity, but also include several other measures, like unprotected sex, to provide additional context.

2.2 Strategies to achieve LGBTQIA+-inclusivity

To ensure LGBTQIA+-inclusivity in the evaluation of SafeSpace, we used the following strategies: (1) staffing the project team with LGBTQIA+ researchers; (2) securing an IRB waiver of parental consent; (3) utilizing an LGBTQIA+focused recruitment approach and (4) developing and refining survey measures to reflect the experiences of LGBTQIA+ youth. To achieve the latter, we gathered feedback from equity experts, conducted cognitive interviews with LGBTQIA+ youth, and secured permission from the funder to omit required measures deemed non-inclusive. Together these strategies embed LGBTQIA+ equity at each stage of the evaluation, increasing the likelihood that LGBTQIA+ youth participants feel included, respected, and represented in this evaluation and enhancing our ability to document program impacts for this population.

2.2.1 Staffing the project with LGBTQIA+ researchers

As a first step towards ensuring that LGBTQIA+ equity was central to the evaluation, we staffed the project with researchers who were members of the LGBTQIA+ community. By intentionally including LGBTQIA+ staff members, we ensured that our team had a critical eye towards LGBTQIA+ youth’s needs and perceptions and were able to identify what was needed to meet those needs throughout the evaluation. Additionally, these staff brought their knowledge as members of the LGBTQIA+ community, as well as credentials related to SRH, LGBTQIA+ psychology, gender and sexuality studies, and LGBTQIA+ health. While the impact of their insights on this project is immeasurable, these staff identified the necessity of each of the following steps taken to increase inclusivity.

2.2.2 Securing IRB approval for consent procedures

Recognizing that requiring parental consent for study participation has unique implications for LGBTQIA+ youth, we requested and received a waiver of parental consent from the Child Trends IRB. We explained that requiring parental consent could result in unwanted disclosure about sexual and gender identity that could contribute to emotional distress for LGBTQIA+ youth. Further, sampling bias would increase if we only included youth with parental support, i.e., those with parental consent.

2.2.3 Utilizing a LGBTQIA+ focused recruitment approach

To recruit LGBTQIA+ youth in our pilot study, we utilized two major social media platforms, TikTok and Instagram, to run paid advertisements (ads). Before launching the pilot, we conducted tests on four platforms—Facebook, Snapchat, TikTok, and Instagram. These tests aimed to identify the most effective targeting parameters, platforms, assets, keywords, and ad formats (such as Reels/Videos, Stories, and Feed) for reaching and engaging LGBTQIA+ youth and youth of color.

2.2.4 Developing and refining LGBTQIA+-inclusive survey measures

To develop LGBTQIA+-inclusive survey measures, we incorporated feedback from the evaluation team, Child Trends LGBTQIA+ equity experts, and LGBTQIA+ youth. We used an iterative process to build consensus among these parties. The evaluation team led the initial measure development and refinement, specifically sexual orientation, gender identity, sexual behavior, and sexual agency measures. Relying heavily on insights from LGBTQIA+ staff, we first identified which questions needed revision or expansion. From there, we integrated findings of published research from The Trevor Project and other equity-focused institutions, including Child Trends’ resources on equity-centered survey design (33–38). We then proposed new measures and shared them with the LGBTQIA+ equity experts.

2.3 Equity experts

Two Child Trends LGBTQIA+ equity experts reviewed the new survey measures for inclusion, clarity, and respectfulness. The experts provided their feedback and suggested edits to the measures in written format or through video calls. The evaluation team applied the agreed upon edits to the survey and began cognitively testing the revised measures with LGBTQIA+ youth.

2.4 Cognitive interviews

We conducted cognitive interviews with seven LGBTQIA+ youth ages 14–18 who were recruited through social media advertisements testing (described above). Interviews were conducted via phone and lasted about 1 h. We presented interviewees with the newly developed questions, and they were encouraged to read the questions to themselves, describe their reactions, and discuss what they might consider when answering each question. Through this process, youth provided feedback and offered suggestions for clarity, comfort, and inclusiveness. In the few cases that feedback from youth contradicted feedback from the equity reviewers we prioritized feedback from youth after consulting with the evaluation team’s LGBTQIA+ staff.

2.5 Sex performance measure

This evaluation is federally funded by the Family and Youth Services Bureau’s Personal Responsibility Education Program—Innovative Strategies (PREIS), and as a PREIS grantee, we are required to collect specific demographic measures for performance measure reporting. We requested a waiver of the required measure for biological sex, which only has “Male” and “Female” response options. Child Trends IRB would not approve the survey instrument without including ‘Intersex’ as a response option, which was a compelling justification for our waiver.

3 Results

3.1 Consent procedures

The specific language used to obtain a parental consent waiver for minor participants is provided in Supplementary Material B. We described how requiring parental consent could result in unwanted disclosure of young people’s sexual or gender identity to parents, potentially resulting in emotional distress and undermine the scientific validity of the evaluation by contributing to selection bias. We also emphasized that (1) participating in SafeSpace involved no more than minimal risk to participants; (2) waiving parental consent did not adversely affect participants’ rights and welfare; (3) youth were required to assent/consent to each of the study requirements before they were enrolled in the study; and (4) youth were given the opportunity to skip intervention content and had access to resources pertaining to each of the curriculum topics after engaging with the material. Through this request, the evaluation team successfully obtained a parental consent waiver for participants ages 14–17. With this consent waiver, we helped to protect LGBTQIA+ participants by preventing potential identity disclosures.

3.2 Recruitment

Pre-pilot testing of ads informed the minimalistic design used for the pilot study recruitment campaign and helped us tailor ads to resonate with the differing experiences, identities, and interests of LGBTQIA+ youth by using LGBTQIA+ pride symbols, color palettes, and themes (see Supplementary Material A).

Pilot study ads performed well, particularly on Instagram. In total, we reached 469,878 youth⁶, resulting in 1,933 link clicks and 801,468 impressions. In total, 42 participants were enrolled in the pilot study, with more than half (61.9%) identifying as LGBTQIA+ .

3.3 LGBTQIA+-inclusive measures

The final measures fall within two domains: demographic characteristics (sexual orientation, gender identity, and sex assigned at birth) and sexual behaviors (sexual touching, rubbing genitals, oral sex, PV sex, anal sex, and sexting). In total, we included three demographic characteristic measures assessing sexual orientation, gender identity, and sex assigned at birth and 13 measures addressing lifetime and recent (past three months) sexual behaviors (Table 1). For each type of measure, we present themes from the internal equity reviews and cognitive interviews as well as baseline prevalence estimates from the pilot study (Table 2).

Table 1

Table 1. Newly developed LGBTQIA+-inclusive measures.

Table 2

Table 2. Feedback for LGBTQIA+-inclusive measures.

3.3.1 Demographic characteristic measures

3.3.1.1 Themes from internal equity reviews

Our equity reviewers provided feedback that incorporated best practices from the literature and examples from well-respected organizations in the field (Table 2). Themes included (1) shifting away from binary measures of sex, gender identity, and sexual orientation and (2) enhancing youth comprehension. Regarding gender, an equity reviewer stated, “there is some evidence that trans folks will choose their sex here accidentally, and I think your sample is old enough to understand girl/woman and boy/man, so I would drop the female and male.” To improve our sexual orientation question, reviewers advised we use question wording from the Trevor Project’s definition of sexual orientation (38).

They also suggested that the parenthetical “not gay” addition to the “straight” response option “may be a holdover from previous times and less necessary now” and that “it also feels like it forces a binary.” Lastly, the reviewers recommended adding write-in options to allow participants to describe their identities in their own words as well as a response option of “I don’t know what this question means.”.

3.3.1.2 Themes from cognitive interviews

Young people participating in the cognitive interviews had positive feedback to share about the demographic measures (Table 2). They indicated that the measures were easy to understand, comprehensive, and inclusive. Although one participant suggested the term “sexual identity” might be more accurate than the term “sexual orientation,” all other participants preferred “sexual orientation” so we retained that language.

3.3.1.3 Pilot study baseline results

Participant demographics are shown in Table 3. Similar proportions of participants identified as bisexual, pansexual, or queer (40.5%) and straight (38.1%). About one in ten (11.9%) indicated that they were questioning their sexual identity, and 4.8 percent identified as lesbian or gay, and asexual each. Most participants identified as a girl or woman (85.7%), followed by non-binary, gender nonconforming, or something else (11.9%). One (2.4%) reported questioning their gender identity.

Table 3

Table 3. Pilot study demographics (N = 42).

3.3.2 Sexual behavior measures

3.3.2.1 Themes from internal equity reviews

For the sexual behavior measures, equity reviews provided feedback related to enhancing (1) youth comprehension, (2) consistency, and (3) gender inclusiveness (Table 2). For example, regarding PV sex, equity reviewers noted, “from a say-what-you-mean perspective, penile-vaginal sex is probably what you are talking about. I have seen the term penis/vagina sex used, which might be easier for youth since the adjectives penile and vaginal may be less familiar than the nouns.” Similarly, in response to a measure about vulva-to-vulva sex, the equity reviewers were “not sure everyone knows the term vulva and apart from this you seem to be using vagina instead, which is probably more commonly understood.” Furthermore, to be more inclusive, they suggested referring this activity as “genital-to-genital sex” when defining non-penetrative types of sex. Additionally, the reviewers noted that specifying that anal sex excludes sex toy use could underestimate the prevalence of this activity given that butt plugs are sometimes used to prepare for anal sex. Reviewers suggested instead that we remove the parenthetical phrase “not a sex toy” from the definition of penetration in anal sex.

3.3.2.2 Themes from cognitive interviews

Themes from the cognitive interviews addressed (1) the comprehensibility and accuracy of the language, and (2) the inclusivity of our language and definitions (Table 2). Some comments helped us to add clarity to our definitions of in-person sexual activity. For example, one participant noted that the phrase “experienced oral sex” could be interpreted as only receiving oral sex and that we should revise it to explicitly measure both receiving and giving oral sex. Other comments affirmed that the measures would be well-understood by young people. For example, regarding our use of both medical and colloquial terminology for genitalia, a participant noted “for some 14-year-olds, [medical terminology] might be confusing to some, particularly anus” but “you put butthole in parentheses, so you’re giving them another word of what it is.”

Young people’s feedback also enhanced the inclusivity of our measures. In particular, one participant highlighted that “for AFAB [assigned female at birth] people who may identify [not as a girl or woman], the word vagina might make them uncomfortable or be triggering. Vulva might be a softer way to phrase it. Vagina [is] also used more as slang [for entire genital area].” This individual suggested using a phrase such as “areas of sexual pleasure,” yet we proposed “private parts” as an alternative to other participants who responded positively to this phrase. Another participant also suggested we alter our language to include asexual youth, noting we should change “wording to [account for] sexual experiences you are willing to do or are comfortable with but not necessarily for pleasure.” Finally, interviewees appreciated the language reminding respondents to report only on wanted sexual experiences, noting that this reminder was “very affirming.”

3.3.2.3 Pilot study baseline results

Overall, 42.9 percent of all participants had engaged in any type of in-person sexual activity in the past three months; participants primarily engaged in sexual touching (40.5%) and oral sex (35.7%); only 4.8 percent engaged in anal sex. Forty-six percent of LGBTQIA+ participants had engaged in any type of in-person sexual activity in the past three months, compared to 37.5 percent of cis-straight participants. Seven percent of LGBTQIA+ participants had anal sex in the past three months whereas no cis-straight participants reported this behavior. Similarly, about one-fifth (19.2%) of LGBTQIA+ participants reported having sex using sex toys whereas no cis-straight participants reported this behavior. None of these differences by LGBTQIA+ status were statistically significant.

Most pilot participants had ever been asked to send a sext (90.4%) and had received a sext (85.4%); more than half had sent a sext themselves (61%) (Table 4). A higher proportion of LGBTQIA+ participants had been asked to sext, received a sext, and sent a sext compared to cis-straight participants yet the differences were not statistically significant.

Table 4

Table 4. Baseline sexual behaviors by LGBTQIA+ Status (N = 42).

We also collected reports of PV sex without contraception and condomless PV or anal sex in the last three months. Less than a quarter of all participants had PV sex without contraception (21.4%) or had engaged in condomless PV or anal sex (23.8%) in the past three months. The differences between cis-straight and LGBTQIA+ participants’ engagement in PV sex without contraception in the past three months was not statistically significant, nor were the differences between participants in rates of condomless PV or anal sex by sexual orientation and gender identity.

4 Discussion

The recent development of innovative LGBTQIA+-inclusive sex education programs helps to fill sexuality education gaps for LGBTQIA+ youth. However, evaluations of these programs must accurately capture the experiences of these populations to ensure LGBTQIA+-inclusive programming is evidence-based and can be widely scaled up. This study shows how we incorporated an integrated approach to develop and pilot test a rigorous LGBTQIA+-inclusive evaluation of the SafeSpace adolescent sexual health program.

Our pilot study demonstrates the feasibility and effectiveness of using social media ads with tailored graphic designs to recruit a sample of youth participants with diverse sexual orientations, racial identities, and gender identities into a mobile app-based intervention. Notably, almost two-thirds of pilot sample participants were LGBTQIA+ and three-quarters were Black and/or Latinx. Because the types of platforms youth use are constantly changing (39, 40), social media recruitment must be tailored to meet the needs of youth. For example, previous studies recruited large samples through Facebook (41–44) and Instagram (33), while this study also used TikTok, which was launched in 2016 (45).

This study also highlights the feasibility of obtaining a parental consent waiver—an important study enrollment practice for evaluation of sex education programs. Parental consent requirements for minors to participate in research can disproportionately exclude LGBTQIA+ youth who have not disclosed their identity to their parents or who are living with chosen family (46). Given that parental consent requirements could lead to selection bias in which LGBTQIA+ youth with parental support are overly represented, we intentionally sought and received exemption from a parental consent requirement by clearly documenting the potential harms to participants and threats to validity.

By incorporating insights from LGBTQIA+ youth, LGBTQIA+ staff, and equity experts, we created survey measures that expand beyond a heteronormative, cisnormative, and risk-oriented prioritization of PV sex (38, 47, 48). These measures better reflect, respect, and affirm the experiences of LGBTQIA+ program participants who may otherwise feel alienated by questions not inclusive of their sexual experiences (49). However, our pilot study baseline findings indicate that these measures better capture the variety of sexual experiences of all youth. Both LGBTQIA+ and non-LGBTQIA+ youth reported engaging in non-PV types of in-person sexual activity, including sexting, sexual touching, genital rubbing, anal sex, and oral sex. As such, we normalize and validate all adolescents’ sexual desires and behaviors. Moving away from measuring only PV sex shifts the adolescent sexual health field from a sex-negative framework focusing on sex that carries a reproductive risk and towards a sex-positive framework that affirms sex for the purpose of pleasure, connection, and wellbeing. which can be developmentally appropriate for adolescents (50). This shift toward sex-positivity allows researchers to document aspects of both LGBTQIA+ adolescents’ and cis-straight adolescents’ sexual wellbeing, not only their sexual health (51).

By including a broader set of measures of sexual behavior, we captured a substantially higher rate of sexually activity from 26 percent who engaged in recent PV sex, to 42 percent who engaged in any type of in-person sexual activity in the past three months. Estimating higher rates of in-person sexual activity also has implications for evaluations’ ability to measure the impact of programming on STI prevention, consent, and other sexual health outcomes. The high rates of PV sex without contraception and condomless PV or anal sex at baseline—21 percent and 24 percent, respectively—reflect substantial exposure to unintended pregnancy and STIs for both LGBTQIA+ and cis-straight participants. Strikingly, almost all youth (both LGBTQIA+ and cis-straight) who reported PV or anal sex at baseline indicated at least one incident of recent sex without contraception or condoms, which points to the need for comprehensive and inclusive sexual health programming.

One notable finding from the pilot was the high percentage of youth engaging in sexting, with 61 percent reporting they ever sent a sexual photo of themselves to another person and 85 percent reporting receipt of sexual photos. These pilot results are much higher than a recent meta-analysis finding that almost 20 percent of youth report sending sexts and more than one-third report receiving sexts (52). The difference in rates may be due to the timeframe (we measured ever vs. recent sexting and measured “sexual photos” instead of “nude photos”).

There are several limitations to this study. First, this is a pilot study so the sample size is small; as such significance testing may be unreliable. Additionally, while we took care to describe how SafeSpace participants would take the evaluation surveys, our cognitive interviewees were presented with the survey items in a word document on a call with study team members, while the participants filled the survey out online independently. This different format may have impacted how they interpreted the questions and responses. Finally, we oversampled LGBTQIA+ youth to understand their unique perspectives and experiences. As such the results are not generalizable. Instead, we hope to demonstrate the feasibility of conducting an LGBTQIA+-inclusive evaluation.

5 Conclusion

Our findings indicate that leveraging the lived experiences of LGBTQIA+ staff and incorporating input from equity reviewers and LGBTQIA+ youth is a feasible and effective approach to designing and implementing LGBTQIA+-inclusive evaluations of sexual health programming. Our study highlights the types of strategies that can successfully be used to conduct LGBTQIA+-inclusive evaluations to better support the sexual health of both LGBTQIA+ and cis-straight youth. Future sexual health program evaluations should incorporate such strategies to inclusively, respectfully, and more accurately capture the experiences and identities of LGBTQIA+ youth. In doing so, program evaluations can document the impact of and need for sexuality education that is inclusive, affirming, and that promotes sexual wellbeing for all.

Data availability statement

The datasets presented in this article are not readily available because we did not receive IRB approval to make the data publicly available. Requests to access the datasets should be directed to Elizabeth Cook, ecook@childtrends.org.

Ethics statement

The studies involving humans were approved by Child Trends Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The Ethics Committee/institutional review board waived the requirement of written informed consent for participation from the participants or the participants’ legal guardians/next of kin because our team wanted to protect the safety of LGBTQIA+ youth, who might not be out to their parents or guardians.

Author contributions

ZB: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing. EP: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing. EC: Conceptualization, Investigation, Methodology, Supervision, Writing – original draft, Writing – review & editing, Project administration. JM: Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing, Project administration. RS: Writing – original draft, Writing – review & editing, Project administration. MC: Writing – original draft, Writing – review & editing. MG: Data curation, Writing – original draft, Writing – review & editing, Methodology. CN: Writing – review & editing. MD: Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article.

Funding for this study was provided by the Department of Health and Human Services, Administration for Children and Families, Family Youth Services Bureau, grant #90AP2697-01-00. Its contents are solely the responsibility of Child Trends and do not necessarily represent the official views of the Department of Health and Human Services, Administration for Children and Families.

Acknowledgments

The authors would like to first thank the seven youth who participated in cognitive interviews and contributed greatly to developing the measures presented. We would also like to thank Ria Shelton at Child Trends for her contributions to the parental consent waiver language submitted to the IRB. We also thank our equity reviewers, Catherine Shaefer and Brandon Stratford at Child Trends, for their review of the survey measures and continued advising on LGBTQIA+-inclusive approaches to the evaluation.

Conflict of interest

MC works at the organization that developed the Real Talk app and was involved in developing the SafeSpace curriculum.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frph.2024.1327980/full#supplementary-material

Footnotes

¹We use the definition of sexuality education that Breuner et al use, which includes content on sexual health topics such as sexually transmitted infections, contraception, sexual activity, and reproductive rights; sexual and romantic relationships; anatomy and reproduction; and personal identity topics such as sexual orientation and gender identity (1).

²LGBTQIA+ is an acronym for lesbian, gay, bisexual, transgender, queer/questioning, intersex, asexual, and other marginalized sexualities and genders. We use the full acronym to reflect the diversity in this population. However, when referring to other programs we follow the acronym used by that organization.

³We chose to include AFAB and intersex participants for several reasons. First, we are interested in understanding contraceptive use as a key outcome, as well as clinic access, which tend to be more relevant to AFAB youth. Additionally, we wanted reliable data on contraceptive use, and male reporting of partner’s contraceptive use is less accurate than method users themselves. Further, as SafeSpace was developed from Real Talk, we aimed for our study population to reflect its users. Over 60% of Real Talk users identify as girl/woman or non-binary/gender non-conforming. Additionally, while there is large variability within the intersex population, we included intersex people in the evaluation to (1) include as many people as possible who may use contraception and be able to get pregnant in our sample without being overly intrusive with questions about their specific condition, and (2) we wanted to include as many LGBTQIA+ people as possible. As such, intersex people are eligible to participate even though some of them are not able to become pregnant.

⁴MyHealthEd, Inc. Real Talk Mobile App Analytics Data. 2019. [Unpublished data].

⁵This SafeSpace program evaluation has an intentional focus on Hispanic and non-Hispanic Black youth, so all other youth are grouped together. We used these categories in our recruitment approach, requiring at least 50 percent of our sample to be Hispanic or non-Hispanic Black youth. We accepted all non-Hispanic, non-Black youth who met the inclusion criteria to make up the rest of the sample.

⁶Reach is a measure used in the social media marketing industry to represent the number of accounts (people) that saw an advertisement at least once.

References

1. Breuner CC, Mattson G, Committee on Adolescence, Committee on Psychosocial Aspects of Child and Family Health, Adelman WP, Alderman EM, et al. Sexuality education for children and adolescents. Pediatrics. (2016) 138(2):e20161348. doi: 10.1542/peds.2016-1348

PubMed Abstract | Crossref Full Text | Google Scholar

2. Haberland N, Rogow D. Sexuality education: emerging trends in evidence and practice. J Adolesc Health. (2015) 56(1):S15–21. doi: 10.1016/j.jadohealth.2014.08.013

PubMed Abstract | Crossref Full Text | Google Scholar

3. Denford S, Abraham C, Campbell R, Busse H. A comprehensive review of reviews of school-based interventions to improve sexual-health. Health Psychol Rev. (2016) 11(1):33–52. doi: 10.1080/17437199.2016.1240625

PubMed Abstract | Crossref Full Text | Google Scholar

4. Chin HB, Sipe TA, Elder R, Mercer SL, Chattopadhyay SK, Jacob V, et al. The effectiveness of group-based comprehensive risk-reduction and abstinence education interventions to prevent or reduce the risk of adolescent pregnancy, human immunodeficiency virus, and sexually transmitted infections: two systematic reviews for the guide to community preventive services. Am J Prev Med. (2012) 42(3):272–94. doi: 10.1016/j.amepre.2011.11.006

PubMed Abstract | Crossref Full Text | Google Scholar

5. Goldfarb ES, Lieberman LD. Three decades of research: the case for comprehensive sex education. J Adolesc Health. (2021) 68(1):13–27. doi: 10.1016/j.jadohealth.2020.07.036

PubMed Abstract | Crossref Full Text | Google Scholar

6. Committee on Adolescent Health Care. Comprehensive Sexuality Education. The American College of Obstetricians and Gynecologists; Report No.: 678. Available online at: https://www.acog.org/clinical/clinical-guidance/committee-opinion/articles/2016/11/comprehensive-sexuality-education (accessed September 26, 2023).

Google Scholar

7. Centers for Disease Control and Prevention. 2021 Number and percentage of students, by sexual identity — United States and selected U.S. sites. (2023 Apr) (cited Sep 27, 2023). Available online at: https://www.cdc.gov/healthyyouth/data/yrbs/supplemental-mmwr/students_by_sexual_identity.htm

Google Scholar

8. Kosciw J, Clark C, Truong N, Zongrone A. The 2019 National School Climate Survey: The Experiences of Lesbian, Gay, Bisexual, Transgender, and Queer Youth in Our Nation’s Schools. Gay, Lesbian, and Straight Education Network. (2020) (cited Sep 26, 2023). Available online at: https://www.glsen.org/sites/default/files/2021-04/NSCS19-FullReport-032421-Web_0.pdf

Google Scholar

9. Rabbitte M. Sex education in school, are gender and sexual minority youth included?: a decade in review. Am J Sex Educ. (2020) 15(4):530–42. doi: 10.1080/15546128.2020.1832009

PubMed Abstract | Crossref Full Text | Google Scholar

10. American Psychological Association. Psychology Topics. (cited Sep 26, 2023). LGBTQ+ Inclusive Curricula: School Curricula Inclusive of LGBTQ+ History, Culture, and People; Sex Education. Available online at: https://www.apa.org/topics/lgbtq/lgbtq-inclusive-curricula

Google Scholar

11. Bible J, Kaplan A, Lieberman L, Goldfarb E. A retrospective analysis of sex education messages received by LGB youth. J LGBT Youth. (2022) 19(3):287–306. doi: 10.1080/19361653.2020.1819509

Crossref Full Text | Google Scholar

12. Hobaica S, Kwon P. “This is how you hetero:” sexual minorities in heteronormative sex education. Am J Sex Educ. (2017) 12(4):423–50. doi: 10.1080/15546128.2017.1399491

Crossref Full Text | Google Scholar

13. Gowen LK, Winges-Yanez N. Lesbian, gay, bisexual, transgender, queer, and questioning youths’ perspectives of inclusive school-based sexuality education. J Sex Res. (2014) 51(7):788–800. doi: 10.1080/00224499.2013.806648

PubMed Abstract | Crossref Full Text | Google Scholar

14. Hobaica S, Schofield K, Kwon P. “Here’s your anatomy…good luck”: transgender individuals in cisnormative sex education. Am J Sex Educ. (2019) 14(3):358–87. doi: 10.1080/15546128.2019.1585308

Crossref Full Text | Google Scholar

15. Paschen-Wolff MM, Kelvin EA, Wells BE, Grov C. Sexual orientation and pregnancy among adolescent women in the national survey of family growth, 2002–2015. LGBT Health. (2018) 5(8):449. doi: 10.1089/lgbt.2018.0036

PubMed Abstract | Crossref Full Text | Google Scholar

16. Charlton MBM, Corliss HL, Missmer SA, Rosario M, Spiegelman D, Austin SB. Sexual orientation differences in teen pregnancy and hormonal contraceptive use: an examination across two generations. Am J Obstet Gynecol. (2013) 209(3):204.e1. doi: 10.1016/j.ajog.2013.06.036

PubMed Abstract | Crossref Full Text | Google Scholar

17. Lindley LL, Walsemann KM. Sexual orientation and risk of pregnancy among New York city high-school students. Am J Public Health. (2015) 105(7):1379. doi: 10.2105/AJPH.2015.302553

PubMed Abstract | Crossref Full Text | Google Scholar

18. Riskind RG, Tornello SL, Younger BC, Patterson CJ. Sexual identity, partner gender, and sexual health among adolescent girls in the United States. Am J Public Health. (2014) 104(10):1957. doi: 10.2105/AJPH.2014.302037

PubMed Abstract | Crossref Full Text | Google Scholar

19. Veale J, Watson RJ, Adjei J, Saewyc E. Prevalence of pregnancy involvement among Canadian transgender youth and its relation to mental health, sexual health, and gender identity. Int J Transgend. (2016) 17(3–4):107–13. doi: 10.1080/15532739.2016.1216345

PubMed Abstract | Crossref Full Text | Google Scholar

20. Grollman EA. Sexual health and multiple forms of discrimination among heterosexual youth. Soc Probl. (2017) 64(1):156–75. doi: 10.1093/socpro/spw031

Crossref Full Text | Google Scholar

21. Charlton BM, Roberts AL, Rosario M, Katz-Wise SL, Calzo JP, Spiegelman D, et al. Teen pregnancy risk factors among young women of diverse sexual orientations. Pediatrics. (2018) 141(4):e20172278. doi: 10.1542/peds.2017-2278

PubMed Abstract | Crossref Full Text | Google Scholar

22. Garcia J, Vargas N, Clark JL, Magaña Álvarez M, Nelons DA, Parker RG. Social isolation and connectedness as determinants of well-being: global evidence mapping focused on LGBTQ youth. Glob Public Health. (2020) 15(4):497–519. doi: 10.1080/17441692.2019.1682028

PubMed Abstract | Crossref Full Text | Google Scholar

23. Blake S, Ledsky R, Lehman T, Goodenow C, Sawyer R, Hack T. Preventing sexual risk behaviors among gay, lesbian, and bisexual adolescents: the benefits of gay-sensitive HIV instruction in schools. Am J Public Health. (2001) 91(6):940–6. doi: 10.2105/AJPH.91.6.940

PubMed Abstract | Crossref Full Text | Google Scholar

24. Rose ID, Lesesne CA, Sun J, Johns MM, Zhang X, Hertz M. The relationship of school connectedness to adolescents’ engagement in co-occurring health risks: a meta-analytic review. J Sch Nurs. (2022) 40(1):10598405221096802. doi: 10.1177/10598405221096802

Crossref Full Text | Google Scholar

25. Miller BC, Benson B, Galbraith KA. Family relationships and adolescent pregnancy risk: a research synthesis. Dev Rev. (2001) 21(1):1–38. doi: 10.1006/drev.2000.0513

Crossref Full Text | Google Scholar

26. Noll JG, Guastaferro K, Beal SJ, Schreier HMC, Barnes J, Reader JM, et al. Is sexual abuse a unique predictor of sexual risk behaviors, pregnancy, and motherhood in adolescence? J Res Adolesc. (2019) 29(4):967–83. doi: 10.1111/jora.12436

PubMed Abstract | Crossref Full Text | Google Scholar

27. Mustanski B, Greene GJ, Ryan D, Whitton SW. Feasibility, acceptability, and initial efficacy of an online sexual health promotion program for LGBT youth: the queer sex ed intervention. J Sex Res. (2015) 52(2):220–30. doi: 10.1080/00224499.2013.867924

PubMed Abstract | Crossref Full Text | Google Scholar

28. Ybarra M, Rosario M, Saewyc E, Goodenow C, Dunsiger S. One-year follow-up after a pregnancy prevention intervention for LGB+ teens: an RCT. Pediatrics. (2023) 151(4):e2022059172. doi: 10.1542/peds.2022-059172

PubMed Abstract | Crossref Full Text | Google Scholar

29. Real Talk. (cited Sep 27, 2023). Real Talk. Available online at: https://myhealthed.org

30. Leos C, Chen E, Jagannathan V. Using human-centered design strategies to identify unmet adolescent sexual health wants and needs. Prev Sci. (2023) 24(Suppl 2):174–84. doi: 10.1007/s11121-023-01559-5

PubMed Abstract | Crossref Full Text | Google Scholar

31. Hamilton BE, Rossen L, Branum A. Teen Birth Rates for Urban and Rural Areas in the United States, 2007–2015. National Center for Health Statistics: Centers for Disease Control and Prevention. (2016). Report No.: 264. Available online at: https://www.cdc.gov/nchs/products/databriefs/db264.htm#Suggested (accessed September 27, 2023).

Google Scholar

32. Tollestrup J. Teen Birth Trends: In Brief. Congressional Research Service. (2022 Sep) (cited Oct 11, 2023). Report No.: R45184. Available online at: https://crsreports.congress.gov/product/pdf/R/R45184

Google Scholar

33. Manlove J, Cook E, Whitfield B, Johnson M, Martínez-García G, Garrido M. Short-term impacts of pulse: an app-based teen pregnancy prevention program for black and latinx women. J Adolesc Health. (2020) 66(2):224–32. doi: 10.1016/j.jadohealth.2019.08.017

PubMed Abstract | Crossref Full Text | Google Scholar

34. Upadhyay UD, Danza PY, Neilands TB, Gipson JD, Brindis CD, Hindin MJ, et al. Development and validation of the sexual and reproductive empowerment scale for adolescents and young adults. J Adolesc Health. (2021) 68(1):86–94. doi: 10.1016/j.jadohealth.2020.05.031

PubMed Abstract | Crossref Full Text | Google Scholar

35. Courtice EL, Czechowski K, Noorishad PG, Shaughnessy K. Unsolicited pics and sexual scripts: gender and relationship context of compliant and non-consensual technology-mediated sexual interactions. Front Psychol. (2021) 12. doi: 10.3389/fpsyg.2021.673202

PubMed Abstract | Crossref Full Text | Google Scholar

36. Manlove J, Parekh J, Whitfield B, Griffith I, Garg A, Fasula AM. A mixed-methods pilot evaluation of manhood 2.0, a program to reduce unintended pregnancy among young men. Am J Mens Health. (2022) 16(3):15579883221104895. doi: 10.1177/15579883221104895

PubMed Abstract | Crossref Full Text | Google Scholar

37. Manlove J, Welti K, Whitfield B, Faccio B, Finocharo J, Ciaravino S. Impacts of Re:MIX-A school-based teen pregnancy prevention program incorporating young parent coeducators. J Sch Health. (2021) 91(11):915–27. doi: 10.1111/josh.13078

PubMed Abstract | Crossref Full Text | Google Scholar

38. DeChants J, Green A, Price M, Davis C. Measuring Youth Sexual Orientation and Gender Identity. The Trevor Project. (2021). (cited Sep 27, 2023). Available online at: https://www.thetrevorproject.org/research-briefs/measuring-youth-sexual-orientation-and-gender-identity/

Google Scholar

39. Vogels E, Gelles-Watnick R, Massarat N. Teens, Social Media and Technology 2022. Pew Research Center: Internet, Science & Tech. (2022) (cited Sep 27, 2023). Available online at: https://www.pewresearch.org/internet/2022/08/10/teens-social-media-and-technology-2022/

Google Scholar

40. Hamilton JL, Dreier MJ, Boyd SI. Social media as a bridge and a window: the changing relationship of adolescents with social media and digital platforms. Curr Opin Psychol. (2023) 52:101633. doi: 10.1016/j.copsyc.2023.101633

PubMed Abstract | Crossref Full Text | Google Scholar

41. Whitaker C, Stevelink S, Fear N. The use of facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res. (2017) 19(8):e290. doi: 10.2196/jmir.7071

PubMed Abstract | Crossref Full Text | Google Scholar

42. Amon KL, Campbell AJ, Hawke C, Steinbeck K. Facebook as a recruitment tool for adolescent health research: a systematic review. Acad Pediatr. (2014) 14(5):439–447.4. doi: 10.1016/j.acap.2014.05.049

PubMed Abstract | Crossref Full Text | Google Scholar

43. Prescott TL, Phillips Ii G, DuBois LZ, Bull SS, Mustanski B, Ybarra ML. Reaching adolescent gay, bisexual, and queer men online: development and refinement of a national recruitment strategy. J Med Internet Res. (2016) 18(8):e200. doi: 10.2196/jmir.5602

PubMed Abstract | Crossref Full Text | Google Scholar

44. Mamey MR, Schrager SM, Rhoades H, Goldbach JT. Nominal versus realized costs of recruiting and retaining a national sample of sexual minority adolescents in the United States: longitudinal study. J Med Internet Res. (2023) 25:e36764. doi: 10.2196/36764

PubMed Abstract | Crossref Full Text | Google Scholar

45. MacKinnon KR, Kia H, Lacombe-Duncan A. Examining TikTok’s potential for community-engaged digital knowledge mobilization with equity-seeking groups. J Med Internet Res. (2021) 23(12):e30315. doi: 10.2196/30315

PubMed Abstract | Crossref Full Text | Google Scholar

46. Cwinn E, Cadieux C, Crooks CV. Who are we missing? The impact of requiring parental or guardian consent on research with lesbian, gay, bisexual, trans, two-spirit, queer/questioning youth. J Adolesc Health. (2021) 68(6):1204–6. doi: 10.1016/j.jadohealth.2020.07.037

PubMed Abstract | Crossref Full Text | Google Scholar

47. The White House. Recommendations on the Best Practices for the Collection of Sexual Orientation and Gender Identity Data on Federal Statistical Surveys. (2022). Report No.: Executive Order 14075. Available online at: https://www.whitehouse.gov/wp-content/uploads/2023/01/SOGI-Best-Practices.pdf (accessed January 11, 2023).

Google Scholar

48. Federal Evidence Agenda on LGBTQI+ Equity. Executive Office of the President of the United States; (2023 Jan) (cited Sep 27, 2023). Available online at: https://www.whitehouse.gov/wp-content/uploads/2023/01/Federal-Evidence-Agenda-on-LGBTQI-Equity.pdf

Google Scholar

49. Parekh J, Ciaravino S, Welti K, Ragonese C, Lapointe L, Manlove J, et al. Equity-focused strategies in a federally funded evaluation of a sexual health program. Child Trends. (2023). doi: 10.56417/4593q1135m

Crossref Full Text | Google Scholar

50. Harden KP. A sex-positive framework for research on adolescent sexuality. Perspect Psychol Sci. (2014) 9(5):455–69. doi: 10.1177/1745691614535934

PubMed Abstract | Crossref Full Text | Google Scholar

51. Mitchell KR, Lewis R, O’Sullivan LF, Fortenberry JD. What is sexual wellbeing and why does it matter for public health? Lancet Public Health. (2021) 6(8):e608–13. doi: 10.1016/S2468-2667(21)00099-2

PubMed Abstract | Crossref Full Text | Google Scholar

52. Mori C, Park J, Temple JR, Madigan S. Are youth sexting rates still on the rise? A meta-analytic update. J Adolesc Health. (2022) 70(4):531–9. doi: 10.1016/j.jadohealth.2021.10.026

PubMed Abstract | Crossref Full Text | Google Scholar