A comparison of applicant and accepted student characteristics to research training programs with implications for recruitment and selection strategy

Introduction: Very few studies have examined the relationship between student characteristics and their acceptance to research training programs that use holistic selection. The present study addressed this question using institutional and applicant data of three NIH undergraduate training programs at California State University, Long Beach. Its first aim was to examine whether the applicants to the training programs were representative of the broader campus population. Its second aim was to investigate whether applicants who were accepted to the programs using a holistic selection process differed in academic discipline, demographics, and psychosocial characteristics from applicants who were not accepted.

Methods: Information on students’ majors, race/ethnicity, and gender was obtained from the university records or applications submitted by students. Majors were categorized as either biomedical or behavioral disciplines, while URM status was defined as students who self-identified their race and ethnicity as African American/Black, Native American, Native Hawaiian/Pacific Islander, or Hispanic. Applicants’ psychosocial characteristics were obtained from a separate online survey. The acceptance status of applicants was obtained from the training programs’ records.

Results: The applicant and non-applicant groups showed similar distribution of demographic characteristics regarding URM status and gender. Moreover, students’ academic discipline and other demographic variables were not associated with application status at either the lower division (LD) or upper division (UD) levels. Although psychosocial characteristics measured with the online survey were not considered in the selection process, post-hoc analyses showed that LD applicants with higher grit and UD applicants with higher science interests were more likely to be accepted to the programs.

Conclusion: The equal representation of URM and female students in the applicant and non-applicant groups suggests that students from these traditionally underrepresented groups in STEM were just as likely to apply to our training programs. Furthermore, while the holistic selection process resulted in comparable acceptance rates across URM status and gender, it appeared to favor LD applicants with higher grit and UD students with higher science interests. These findings imply that research training programs can effectively recruit diverse students from underrepresented populations in STEM by using intentional outreach and recruitment efforts coupled with an objective and holistic selection process.

1 Introduction

Recent trends in higher education call for increased spending and focus on developing and producing greater numbers of higher numbers of graduates in Science, Technology, Engineering and Math (STEM) fields. While there has been an increase in high school graduation of underrepresented minority students who enter college and an increase in those students who complete their degree in the field of science and pursue advanced degrees, their numbers are still behind those of well-represented students (Hurtado et al., 2008; National Science Foundation, 2024). The smaller number of URM students in the field of science results in fewer researchers who are more likely to address the issues of under-represented populations in health-related fields (Hurtado et al., 2008). Fortunately, research on undergraduates in STEM shows that substantial involvement in faculty-led research, particularly at an early stage of undergraduate studies, improves students’ retention and graduation rates and pursuit of careers in science, including for students from groups historically underrepresented in STEM (Gilmore et al., 2015; Hernandez et al., 2018; Hurtado et al., 2008). Additionally, intensive undergraduate research training has been found to be effective both at the lower and upper division levels in increasing students’ research knowledge and skills, science identity, and matriculation in graduate programs (Eagan et al., 2013; Hall et al., 2016; Hurtado et al., 2017; Kingsford et al., 2022; Vu et al., 2023). More importantly, students from historically underrepresented groups in STEM appear to benefit from such training programs as much as those from well-represented groups (e.g., Estrada et al., 2016; Hall et al., 2016; Vu et al., 2023). However, little is known about who applies to these undergraduate research training programs. Underrepresentation can occur at different points along the pipeline. To fill this gap, the present study examined the rates of application and acceptance to federally funded undergraduate research programs at California State University, Long Beach (CSULB), a Hispanic-Serving Institution (HSI) and Asian American Native American Pacific Islander-Serving Institution (AANAPISI), across several relevant academic disciplines, demographic, and psychosocial characteristics. Such information can help shed light on what might contribute to the continued underrepresentation of students from historically disadvantaged groups.

In response to a critical need to diversify the nation’s biomedical research workforce and eliminate health disparities, the National Institutes of Health (NIH) has, for decades, funded training programs that support the development of historically underrepresented students (e.g., African American, Latine, Native American, Pacific Islander, and women) pursuing research careers in health-related disciplines. The charge for programs such as the Minority Biomedical Research Support (MBRS) and Minority Access to Research Careers (MARC) was to “provide research training opportunities to students and faculty from minority groups underrepresented in the biomedical and behavioral sciences relevant to biomedicine” (e.g., NIH PAR 99-091, 1999). However, the climate for such minority research training programs was forced to change beginning in the mid-1990s with the passing of California Proposition 209 in 1996 that banned affirmative action policies in the state of California and the U.S. fourth Circuit Court of Appeals decision in the 1994 case of Podberesky v. Kirwan that ruled against the University of Maryland’s scholarship program that was exclusively for African American students. When race and gender-based admission became illegal, programs that limited the eligibility to only historically underrepresented groups had to open their programs to non-minority students (Mervis, 2003). Eventually, in 2016, the NIH changed the name of MARC from “Minority Access to Research Careers” to “Maximizing Access to Research Careers” to emphasize its broadened mission to diversify the biomedical research workforce that goes beyond the four historically underrepresented minority ethnic groups.

While targeted outreach and recruitment for students from underrepresented groups is still legal and required by these federally funded programs, the impact of race/ethnicity and gender-neutral selection on diversity of applicant and trainee pools was feared to be devastating (Mervis, 2003). To ensure the trainee pool is diverse, training programs began to adopt a holistic review process of applications that goes beyond traditional measures of academic achievement and potential, such as GPA and standardized test scores. Non-academic factors such as an applicant’s life experiences that demonstrate a genuine interest and motivation in a biomedical research career and bring diverse perspectives to research (Sedlacek, 2005) began to carry increased weight in evaluating applications and selection interviews. While there is no consensus yet on how to conduct a holistic review, early research on holistic admission processes shows promising results, linking it to greater diversity in the admission pools of graduate and professional degree programs (e.g., Henderson et al., 2023; Lewis et al., 2021) than using the traditional metrics. However, we are unaware of any research on the impact of holistic selection processes on the characteristics of accepted applicants to undergraduate research training programs.

The importance of psychosocial characteristics has been reported in higher education literature (Dweck, 2006; Eagan et al., 2023; Miller and Orsillo, 2020; Radunzel et al., 2016; Ramos et al., 2024). In the context of admission, these non-academic traits serve as important indicators of psychological resilience that can foster (and thus predict) success for students from historically underrepresented groups in the graduate and research pipeline in STEM. Grit defined as the combination of passion and perseverance for long-term goals (Duckworth and Quinn, 2009) has been linked with school motivation, academic conscientiousness, and academic performance (Christopoulou et al., 2018). Studies on growth mindset found it to be beneficial for developing resilience when faced with challenging school transitions and challenging math courses (Yeager and Dweck, 2012). Science interest has been linked with persistence and completion of a STEM degree (Radunzel et al., 2016) while general and researcher self-efficacy predicted many research related concepts, including intention to pursue research, research identity, and research productivity (Chemers et al., 2011; Frantz et al., 2017; Livinƫi et al., 2021). Students majoring in health and natural sciences also appeared to be more motivated by intrinsic work values than those in other majors (Balsamo et al., 2013; Johnson and Elder, 2002).

In sum, the present study attempts to fill the gap in the literature by studying the characteristics of students who applied to the NIH-funded undergraduate research training programs at CSULB. We sought to answer the following three broad questions:

1. Are the applicants representative of our diverse student population on academic discipline and demographic variables associated with historical underrepresentation in STEM?

2. Are the students accepted into the training programs representative of the applicant pool on academic discipline and demographic variables associated with historical underrepresentation in STEM?

3. What psychosocial characteristics of applicants are being captured by our holistic selection process that are predictive of acceptance to the training programs?

2 Methods

2.1 Training programs

The three NIH-funded undergraduate research training programs were the BUilding Infrastructure Leading to Diversity (BUILD), Maximizing Access to Research Careers Undergraduate Student Training in Academic Research (MARC U*STAR), and Research Initiative for Student Enhancement (RISE) Programs. While the three programs shared the mission of the NIH to enhance the diversity of our nation’s health-related research workforce, the targeted disciplines and student populations varied somewhat. The BUILD Program was a scaled-up program with the largest number of trainees (averaging around 130 per year) and offered training to students from both biomedical and behavioral sciences at both lower-division (LD) and upper-division (UD) levels. The LD program was designed as an early research intervention targeting sophomore-level students who are interested in exploring research. The UD program targeted juniors and seniors who were interested in obtaining intensive research training designed to make them more prepared and competitive for entry into Ph.D. programs in health-related disciplines. The MARC U*STAR Program (10 trainees per year) aimed to develop a diverse pool of honors undergraduates who want to pursue a Ph.D. or M.D./Ph.D. in biomedical and behavioral fields. MARC U*STAR only offered UD training. The RISE Program (20–25 students per year) trained students also in biomedical and behavioral fields and at LD and UD levels. While all three programs accepted trainees in behavioral fields, BUILD had a broader definition of eligible behavioral disciplines as well as the largest numbers of behavioral trainees. It also engaged in more intentional outreach and recruitment from the behavioral sciences. These three training programs were implemented separately with their own grant funding and research training curriculum. However, because of the shared overall mission, a joint application was developed and implemented to minimize the burden on students to apply to multiple programs, allow the program directors to identify which program would be most suitable for the accepted students, and enhance administrative efficiency.

2.2 Participants

Participants consisted of two groups: The first group consisted of undergraduate students from the representative majors (see Appendix A) of our training programs (N = 28,020) who first enrolled in the university during the fall semester of 8 recruiting cycles of 2015–2023. The second group consisted of applicants to the three NIH training programs (N = 246 for LD and 805 for UD). LD applicants were sophomore-level students with at least 3 years remaining until graduation who applied to the two Associate Programs (BUILD and RISE) designed to introduce students to research and research careers. Applicants to the LD programs had little or no prior research experience and were interested in learning about research and research careers. UD applicants were juniors and seniors who applied to any or all of the three NIH UD training programs (BUILD, MARC U*STAR, and RISE) that provide intensive faculty-mentored research training and professional development support. Applicants to the UD programs tended to have some prior research experience and a solid interest in pursuing graduate school in a biomedical or behavioral science discipline. Detailed descriptions of the LD and UD BUILD program components are available in Kingsford et al. (2022) and Vu et al. (2023), respectively. More information about the demographic characteristics of the participants can be found in section 2.5: Students’ academic and demographic information (see Supplementary Tables 1–4). The three training programs were continuously funded by the NIH during the data collection phase between 2015 and 2023. The LD applicants were pooled from 4 recruiting cycles (2015–2019) because the BUILD Associate program was discontinued in the Summer of 2019. The UD applicants were pooled from 8 recruiting cycles (2015–2023).

2.3 Outreach and recruitment

The on- and off-campus outreach and recruitment were conducted jointly by the three NIH-funded campus programs. The goals of outreach and recruitment were to raise students’ awareness of undergraduate research opportunities and resources on campus and increase their understanding of the benefits of engaging in research with faculty. The joint efforts were also designed to reduce undergraduates’ confusion about the goals of the three NIH-funded programs and increase administrative efficiency. Ultimately, we wanted to reduce barriers for students by streamlining the application process. To achieve NIH’s goal of increasing the number of URM students pursuing a doctoral degree and research careers in health-related disciplines, we employed intentional strategies to distribute resources to reach students, particularly URM students, from the targeted disciplines. On-campus outreach and recruitment included posting flyers and marketing materials across campus and on the BUILD Program website,¹ hosting multiple in-person and virtual information sessions, and making presentations at classes and student organization meetings, hosting information tables at campus events, and creating and maintaining a campus-wide searchable online faculty research mentor directory that lists mentors from various training programs.

In recognition of the potential barriers, such as mistrust of science, academia, and faculty, intentional efforts were made to reach out to student service centers and student organizations that serve students from minoritized communities. Moreover, the science/engineering curriculum was enhanced to infuse research into lower and upper-division courses and offer new courses on interdisciplinary research on health disparities, scientific research communications, and research methods courses (Arruda et al., 2024; Taing et al., 2022). These multi-pronged approaches were designed to transform the student research culture on campus. The goal was to demystify science and research, highlight the relevance of science and health-related research to their communities, and emphasize the importance of diversifying the research workforce to reduce and eradicate health disparities in our society.

Off-campus outreach and recruitment included posting flyers at local community colleges, hosting information sessions at their campus, and providing online recruitment videos customized for each campus. In addition, application support workshops and office hours were offered to any interested students to assist them with their application preparation. We followed the protocols approved by the university’s Institutional Review Board for obtaining applicant and trainee (i.e., accepted student) data. More details of the outreach and recruitment are described in Kingsford et al. (2022) and Vu et al. (2023).

Applicants’ responses to “How did you hear about our program?” indicated that students heard about the NIH training programs through multiple sources (see Appendix B). The most frequently selected source was “hearing about programs through course instructors or faculty mentor,” followed by flyers/posters, other students, current trainees, and information sessions. Accepted students, particularly URMs, have also anecdotally indicated that the encouragement from their instructors and research mentors was instrumental in their decision to apply to the programs because they did not think they would be qualified for the NIH research programs.

2.4 Selection

The selection committee consisted of faculty program directors and faculty research mentors in the three NIH-funded programs. The application packet consisted of an application form, an academic transcript(s), a personal statement and a research interest statement (see Appendix C for prompts for these statements), and a faculty/teaching assistant recommendation form (one or two recommendation forms for LD and UD applications, respectively). The committee evaluated each student’s application holistically using an evaluation rubric (see Appendix D for the LD rubric and Appendix E for the UD selection rubric): each faculty reviewer evaluated applicants on traditional metrics (i.e., academic record, faculty recommendation, academic and career goals, and interest in health-related research) and non-traditional metrics (i.e., ability to enhance diversity of perspectives among the trainees and resiliency in the face of challenges). Student demographic information was not considered in the review. Instead, this holistic review process was implemented to ensure students from historically underrepresented groups with strong potential in a STEM research career would be identified. To increase reliability, each application was evaluated independently by two faculty members.

2.5 Students’ academic and demographic information

Information on campus population and applicants’ academic majors, race/ethnicity, and gender was obtained from the University records provided by the Institutional Research and Analytics (IR&A) Office. Similar information on applicants who were accepted into the program was obtained from the application forms. Majors (see Appendix A for listing) were further classified as biomedical (natural sciences or engineering), behavioral (clinical, health or social sciences) sciences, or undeclared. Biomedical disciplines included majors in two departments in College of Natural Sciences and Mathematics (Biological Sciences and Chemistry & Biochemistry) and select departments in College of Engineering (e.g., Biomedical Engineering, Chemical Engineering, Electrical Engineering, Mechanical Engineering). Behavioral disciplines include majors in select departments in College of Liberal Arts (e.g., Anthropology, Linguistics, Psychology, Sociology) and in College of Health and Human Services (e.g., Family and Consumer Sciences, Health Care Administration, Health Science, Kinesiology). Official university records were used as the primary source for identifying disciplines, but for non-matriculated applicants (transfer student applicants who were not yet in the university system) we used the data provided in the joint application. To designate historically underrepresented students in STEM and following NIH reporting requirement, applicants were categorized as either URM (i.e., African American/Black, Native American, Native Hawaiian/Pacific Islander, or Hispanic) or non-URM (Asian American, White, or Non-Hispanic multiple races). Gender was coded as male, female, or non-binary.

2.6 Measures of applicants’ psychosocial characteristics

Applicants’ psychosocial characteristics were measured with self-report instruments via an online survey that was sent to students upon receiving their applications. The survey was separate from the application process and was not included in the application materials that were evaluated by the selection committee. The psychosocial characteristics were collected for the purpose of program evaluation and research only and were used in the present study to determine which psychosocial characteristics are predictive of acceptance to the research training programs and thus being captured by our holistic evaluation. Grit was assessed with the 8-item Grit-S scale (Duckworth and Quinn, 2009) that measures the tendency to maintain passion and perseverance in working towards a long-term goal despite challenges and setbacks (e.g., I am a hard worker). The response scale ranged from 1 (does not describe me) to 5 (describes me extremely well). Four items were reverse coded. The mean score of the eight items was computed to represent the Grit composite. Growth mindset was assessed with the 4-item Growth Mindset scale (Midkiff et al., 2018) that measures students’ belief that success can be achieved through hard work and effort rather than fixed intelligence (e.g., No matter who you are, you can significantly change your intelligence level).

The response scale ranged from 1 (strongly disagree) to 5 (strongly agree). The mean score of four items was computed to represent the Growth Mindset composite. Science interest was assessed with 5 items measuring perception about enjoyment of science (e.g., I enjoy figuring out answers to scientific questions) and one item measuring problem-solving strategy (i.e., To understand science, I sometimes think about my personal experiences and relate them to topic of being analyzed). These items were adopted from the Colorado Learning Attitudes about Science Survey developed for use in biology (Semsar et al., 2011). For the current study, the word ‘biology’ was replaced with ‘science’ to measure overall science interest. Response scale ranged from 1 (strongly agree) to 5 (strongly disagree). All the 6 responses were reverse coded to represent higher numbers being indicative of higher science interest, and the mean score of the 6 items was computed. Work value was assessed with 3 items from the Intrinsic Awards subscale (e.g., A job where I have the chance to be curious or creative) of a measure used by Johnson and Elder (2002). Response scale ranged from 1 (not at all important) to 5 (extremely important), and the mean score of the 3 items was computed. Self-efficacy was assessed with the New General Self-Efficacy Scale (NGSE; Chen et al., 2001) that includes 8 items (e.g., When facing difficult tasks, I am certain that I will accomplish them) measuring how much people believe they can achieve their goals, despite difficulties. Applicants were asked to rate their feelings in relation to experiences they have had pursuing college studies on a response scale of 1 (strongly disagree) to 5 (strongly agree). The mean score of the 8 items was computed to create a composite score of Self-Efficacy.

3 Results

3.1 Comparisons of applicants and non-applicants to training programs

The first research question addresses the applicants’ representativeness of the diverse student population on key demographic variables associated with historical underrepresentation in STEM. Intuitively, a direct comparison of applicant demographics to those of the entire campus population would provide the most straightforward test of the applicants’ representativeness. We provide these descriptive values when presenting the results below; however, we are not able to determine whether any observed differences between the applicants and the general student population would be statistically significant as we are not aware of any statistical test that would be appropriate for such a comparison. For instance, a chi-square analysis requires that observations to be classified in one, and only one, category (i.e., assumption of independence). This independence assumption would be violated in this direct comparison because the students who applied to the program are a subset of the general student population. Thus, to address our first research question, we compared the number of students who applied to those who did not apply (i.e., their application status) across academic disciplines (biomedical vs. behavioral), URM status and gender, which allows for the Chi-Square test for Independence. Due to missing values in the demographic data provided by the IR&A office, the analytic sample size of applicants was reduced and varied across Figure 1 and Supplementary Table 1.

Figure 1

Figure 1. Percentages of applied and not-applied students and campus population by (A) academic discipline, (B) URM status, and (C) gender.

Figure 1A presents the percentages of students by academic discipline (biomedical vs. behavioral sciences) for the comparison between the students who applied to the NIH programs and those who did not. There were more biomedical students in the applicant pool (71.35%) compared to the campus population (38.93%). In addition, the Chi-Square test for Independence that examined whether students’ application status is associated with discipline category showed that there was a statistically significant over-representation of biomedical students in the applicant pool (71.35%) compared to the non-applicant pool (37.87%), χ² (1, N = 28,020) = 403.03, p < 0.001. This result is not surprising given that two of the three training programs have long been training students in biomedical disciplines at CSULB and are thus well known among CSULB students in natural sciences.

Figure 1B presents parallel percentages for URM status (URM vs. non-URM). Numerically, there were slightly more URM students (62.71%) in the application pool compared to the campus population (60.86%). However, the Chi-Square test for Independence revealed that the application status distribution did not depend on URM vs. non-URM status, χ² (1, N = 25,656) = 1.13, p = 0.29, meaning the URM percentages among applicants (62.71%) and non-applicants (60.80%) were not significantly different. This pattern was different from the general trends noted in past reports that fewer URM students applied to research programs compared to their non-URM counterparts.

Figure 1C presents parallel percentages for gender (male vs. female). Nonbinary was not included in the analysis because it was not an option available in the IR&A dataset. Numerically, there were slightly fewer females in the applicant pool (63.53%) than in the campus population (64.2%). However, the Chi-Square test for Independence showed that the application status distribution did not depend on gender, χ² (1, N = 27,993) = 0.18, p = 0.68, with 63.53% females in the applicant pool compared to 64.2% females in the non-applicant pool. This suggests that the applicant pool’s gender distribution closely mirrors that of the campus population and thus male and female students were equally likely to apply to the research programs on campus, unlike the general trends previously reported.

3.2 Comparisons of accepted and not-accepted applicants at the lower-division level

To address the second research question about the accepted students’ representativeness of the applicant pool, we compared the distributions of accepted and not-accepted students across the academic disciplines (biomedical vs. behavioral), URM status, and gender and conducted the Chi-Square tests for Independence. These tests were performed separately for the LD and UD applicants: they were reviewed with slightly different criteria given the differential emphasis of the training programs at the LD and UD levels (early exposure for LD and intensive training for UD). We would like to note that, as was the case in section 3.1, we could not conduct a direct comparison between the accepted students and the entire applicant pool (which included the accepted students) because it would violate the independence assumption.

Figure 2 (and Supplementary Table 2) presents the academic discipline and demographic data for the accepted and not-accepted LD applicants and total LD applicants, pooled across the 4 recruiting cycles of 2015–2019 during which the LD programs were implemented. It shows that biomedical students comprised 74.8% of total LD applicants whereas they comprised 73.29% of the accepted LD students. The Chi-Square Test for Independence on the association between acceptance status distribution and academic discipline showed that the distribution of acceptance status did not depend on academic discipline, χ² (1, N = 246) = 0.43, p = 0.51. Next, URMs comprised 45.38% of total LD applicants whereas they comprised 48.98% of the accepted students. However, the Chi-Square Test for Independence on the association between acceptance status distribution and URM status showed that acceptance status distribution did not depend on URM status, χ² (1, N = 249) = 1.87, p = 0.17. Finally, females comprised 60.56% of total LD applicants whereas they comprised 59.46% of the accepted students. Again, the Chi-Square Test for Independence showed that acceptance status distribution did not depend on gender, χ² (1, N = 251) = 0.18, p = 0.67. Taken together, these results indicate that the academic discipline and demographic characteristics of students accepted into the LD program did not differ from that of the applicant pool.

Figure 2

Figure 2. Percentages of accepted and not-accepted LD students and total applicants by (A) academic discipline, (B) URM status, and (C) gender.

Because our program data was more detailed than the data obtained from university records, we were able to disaggregate the race/ethnicity data for the applicants, instead of collapsing them into URM vs. non-URM categories. Figure 3 (and Supplementary Table 3A) presents the detailed race/ethnicity data for students accepted and not accepted into the LD program and the total LD applicants. The majority of LD applicants were Hispanic/Latino (42.67%) followed in the order by Asian American, White, and African American/Black. Similarly, the majority of accepted LD students were Hispanic/Latino (45.32%) followed by Asian American, White and African/Black. The Chi-Square test for Independence on the association between acceptance status and race/ethnicity showed that the acceptance status distribution did not depend on race/ethnicity, χ²(3, N = 232) = 0.18, p = 0.67, implying that the accepted LD students’ ethnicity and race are representative of those of the entire LD applicant pool.

Figure 3

Figure 3. Percentages of accepted and not-accepted students by Race and Ethnicity at (A) LD and (B) UD levels.

3.3 Comparisons of accepted and not-accepted applicants at the upper-division level

In this section we examine the second research question for the UD program applicants. Figure 4 (and Supplementary Table 4) shows the academic discipline and demographic distributions of students who were accepted and not-accepted into the UD programs and the total UD applicants, pooled across the 8 recruiting cycles of 2015–2023. It shows that biomedical students comprised 66.09% of total UD applicants whereas they comprised 64.03% of the accepted UD students. The Chi-Square test for Independence on the association between acceptance status distribution and academic discipline showed that acceptance status distribution did not depend on discipline, χ ²(1, N = 805) = 1.04, p = 0.31. Similarly, URMs comprised 51.44% of total UD applicants whereas they comprised 53.97% of the accepted UD students. The Chi-Square test for Independence on the association between acceptance status distribution and URM status showed that acceptance status distribution did not depend on URM status, χ² (1, N = 799) = 2.37, p = 0.12. In terms of gender, females comprised 62.94% of total UD applicants whereas they comprised 63.95% of the accepted UD students. Again, the Chi-Square test for Independence showed that acceptance status distribution did not depend on gender, χ² (1, N = 804) = 0.41, p = 0.52. Thus, the results of these Chi-Square tests imply that the accepted UD students are representative of the entire UD applicant pool.

Figure 4

Figure 4. Percentages of accepted and not-accepted UD students and total applicants by (A) academic discipline, (B) URM status, and (C) gender.

Figure 3 (and Supplementary Table 3B) presents the race/ethnicity distribution of students accepted and not-accepted into the UD program. The majority of applicants were Hispanic/Latino (46.93%) followed by Asian American, White, and African American/Black. Similarly, the majority of accepted students were Hispanic/Latino (50.61%) followed by Asian American, White and African/Black, suggesting a trend of more Hispanic/Latino students being selected into the training programs. However, the Chi-Square test for Independence on the association between acceptance status and race/ethnicity showed that acceptance status distribution did not depend on race/ethnicity, χ² (3, N = 765) = 5.2, p = 0.16. Again, this finding implies that the accepted UD students’ ethnicity and race are representative of those of the entire UD applicant pool.

3.4 Predictability of acceptance for LD and UD applicants

Our final research question examines whether psychosocial characteristics of student applicants predicted acceptance to the LD versus UD research training programs. Applicants’ data on the psychological measures were not used in the selection process and were used solely for the purpose of program evaluation and research. The purpose of this research question is to determine which psychosocial characteristics known to predict various STEM outcomes, if any, are captured by our holistic selection process. Logistic regression analyses were performed, separately for LD and UD applicants, to examine whether students’ academic discipline and demographic and psychosocial characteristics can uniquely predict applicants’ acceptance to the research training programs while controlling for the rest of characteristics. The alpha level for both analyses was set at 0.05. The means, standard deviations, and correlations among psychosocial predictors are presented in Table 1. For both LD and UD applicants, all psychosocial predictors were moderately correlated with each other (0.2 ≤ r ≤ 0.61, p < 0.001 for all).

Table 1

Table 1. Mean (SD) and correlations among psychosocial characteristics.

For LD applicants, only grit was a significant predictor of acceptance to the LD programs while controlling for other predictors (Table 2) such that students with one higher grit score were 2.24 times more likely to be accepted to a LD program, B = 0.81, Wald = 4.32, p = 0.04, OR = 2.24, 95% CI [1.05, 4.78].

Table 2

Table 2. Demographic and psychosocial characteristics predicting LD program admission.

For UD applicants, only science interest predicted acceptance while controlling for other predictors (Table 3) such that applicants with one higher science interest scores were 1.59 times more likely to be accepted to an UD program, B = 0.47, Wald = 4.96, p = 0.03, OR = 1.59, 95% CI [1.06, 2.40]. Academic discipline predictability approached statistical significance (p = 0.07) such that behavioral, compared to biomedical, applicants were slightly more likely to be admitted to the programs.

Table 3

Table 3. Demographic and psychosocial characteristics predicting UD program admission.

4 Discussion and conclusion

Our campus has three NIH-funded undergraduate research training programs designed to train a diverse group of undergraduate students who intend to pursue Ph.D. degrees and become researchers in health-related disciplines. We examined the demographic and psychosocial characteristics of undergraduate students who applied to these training programs and those that were accepted to the programs. We were concerned as to whether students from historically underrepresented groups were still underrepresented in the applicant pool and in the accepted pool. We used intentional strategies to appeal to URM students in our extensive outreach and recruitment efforts and used a joint application to mitigate the typical barriers URM students might face in applying to multiple research training programs. These efforts were effective as we found comparable application rates for students from historically underrepresented groups in STEM to those from our student population. As shown in Appendix B, students who applied to our program reported that they heard about the program from multiple sources. Moreover, we established a strong undergraduate research curriculum that was designed to change the research culture on campus and increase interest and access to research for our students. Our holistic selection process that could not take race, ethnicity and gender into consideration did not negatively impact the acceptance rates of the URM and female applicants into our programs. We found evidence that our selection rubrics were successful in capturing the psychosocial traits such as grit for LD applicants and science interest for UD applicants that are known to predict the successful student outcomes in STEM.

Past studies found persistent racial/ethnic disparities in pursuing research and Ph.D. programs. Black students reported to be less certain that they would pursue postbaccalaureate degree options than their White counterparts (Carter, 2001), and to be less interested in becoming a researcher in a university (Cole and Barber, 2003). Furthermore, both Black and Latinx students were less interested in research as freshmen because of lower level of SAT math scores and were less likely to be interested in a research job outside of university than Asian and White students (Cole and Barber, 2003). In addition to racial/ethnic disparity in interest in pursuing research, Herber (2018) found that German students who are the first in their family to enter college were less likely to apply for highly selective scholarships. Given the past findings, the non-underrepresentation of the URM and female students in both the overall applicant pool and the accepted student pool indicates that our extensive on-campus and off-campus outreach and recruitment strategies and the holistic selection process that incorporated psychosocial factors of science interest and grit were successful in mitigating barriers, drawing and admitting URM and female students to our research training programs.

Bastedo (2021) indicated that 95% of US universities use holistic evaluation for admissions, but what holistic evaluation meant in practice varied widely: most looked at the “whole file,” while 20% considered the “whole person,” placing emphasis on the applicant’s character, and about 30% used “whole context,” which takes into account the unique contribution of the applicant based on opportunities afforded to them by their socioeconomic and family backgrounds. Bastedo noted that the whole context approach is most ideal but not often used. Our holistic selection process aligns with the whole context approach and has objective evaluation rubric appeared to have been effective in accepting students with traits that are predictive of resilience and persistence in academia and STEM, which should capture students from historically underrepresented backgrounds. Even though applicants’ scores on the psychosocial attributes were not available to the selection committee, LD applicants with higher grit and UD applicants with higher science interest were more likely to be accepted. This result implies that criterion #6 in the selection rubric (i.e., demonstrated resilience in the face of challenge) was able to capture LD students’ grit, acknowledging the life experiences and perspectives of individuals from underrepresented groups. In fact, we found that the acceptance rate into the programs tends to be higher for URM applicants than non-URM applicants at both LD and UD levels. A potential reason that grit was not a significant predictor for UD admission is that UD students who applied to the program might already have high levels of grit. At the UD level, self-selection might have occurred where only those students high in grit applied to the program. Given that grit predicted academic success of university students (Hodge et al., 2017), our result underscores the importance of considering grit in a holistic selection process, particularly for early intervention programs at the lower division level. Science interest was a significant predictor of admission for the UD applicants and was best captured by criterion #4 in the selection rubric (i.e., strong interest in biomedical sciences, a well-articulated research question, and/or prior research). But science interest was not a significant predictor of admission for the LD applicants, which might be because a majority of the applicants may not have developed a scientific mindset or prior research experience during their first year of college. Thus, future selection rubrics might need to adopt broader language in measuring applicants’ science interest (e.g., I think about the science I experience in everyday life) as used in the science interest scale. Taken together, these findings serve as a validation of our holistic selection process and suggestions for modification for future holistic selection process.

5 Limitations and implications

This study has several limitations, most notably in sample size, construct validity, and missing data. First, since the study results were based on the data from one institution, sample size limitations prevent inference to the general student population across all other NIH programs at other institutions. However, CSULB represents one of the largest and most diverse institutions in the nation in terms of student population compared to other NIH program sites. Thus, the results provide important implications for the overall NIH programs. Second, there is a substantial debate on the operational definition of the grit construct such that whether two facets of grit, i.e., passion and perseverance, should be used separately or should be combined to represent grit (Credé, 2018; Guo et al., 2019). Duckworth et al. (2021) maintained that the original definition of grit with the sum of all eight items should be used for studying the construct of grit but suggested that a subset of items can be used when a researcher is particularly interested in each facet of grit. Because we intended to investigate the predictability of the overall grit on the admission to the NIH programs, we used the sum of all eight items. Duckworth et al. (2021) also proposed that the original 12 item-grit scale has better content validity. Future study should use the entire 12 items for studying the predictability of grit on a program admission. Finally, there was a sizable number of students (N = 156 for applied and N = 2,208 for not-applied groups) whose URM status was unknown from university records (e.g., they did not disclose their race or were international students). But the number of unknown/declined for URM status was less than 2 for program applicants, for which URM status can be known through self-disclosure in the application. Therefore, it does not seem that students were reluctant to answer race/ethnicity questions in their program application. The unknown URM status in the institutional data appears to reflect factors unrelated to students’ willingness to disclose their racial and ethnic background.

Despite these limitations, our findings have significant implications for future scholarships and research training programs on a local and national scale. The NIH has long funded undergraduate programs dedicated to increasing the diversity of student researchers in health-related disciplines. While it aims to aggressively tackle the persistent racial/ethnic and gender disparities in biomedical and behavioral research workforce, it has placed strict non-exclusionary policy on the selection of research trainees. Our findings are particularly important given the uncertain landscape regarding consideration of race in the U.S. college admission, including the ruling from the U.S. Supreme Court’s in 2023 that effectively overturned the use of race-conscious policies at Harvard University and the University of North Carolina at Chapel Hill. Specifically, they demonstrate that diversification of the biomedical research workforce can be achieved through the adoption of a transparent and objective holistic selection process that uses a standardized rubric that does not explicitly consider race. Proper use of this approach can help administrators avoid the historical misapplication of holistic admission processes that resulted in exclusion, rather than inclusion, of individuals from specific racial groups in the admission process decades ago (Harpalani, 2023; Wechsler, 1984). We believe that employing systematic and extensive outreach and recruitment, investing in undergraduate research curriculum, establishing a strong student-centered research culture on campus, and utilizing a transparent and objective holistic selection process that capture evidence of psychosocial characteristics predictive of resilience and persistence in STEM have together contributed to the equal representation of students from historically underrepresented groups in our campus research pipeline. Further empirical research is needed to verify our interpretation.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by Institutional Review Board at the California State University, Long Beach. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

Y-HC: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing – original draft, Writing – review & editing. C-AC: Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Writing – original draft, Writing – review & editing. HR: Data curation, Methodology, Writing – original draft, Writing – review & editing. PB: Data curation, Funding acquisition, Project administration, Writing – review & editing. VN: Data curation, Funding acquisition, Project administration, Writing – review & editing. K-PV: Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Numbers UL1GM118979, TL4GM118980, RL5GM118978, R25GM071638, and T34GM008074. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Acknowledgments

We would like to thank Ashley Colburn for data preparation, Nicole Streicker for data collection, and Stefania Za for assistance with manuscript formatting.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2025.1474591/full#supplementary-material

Footnotes

1. ^https://www.csulb.edu/BUILD

References

Arruda, E., Zavala, A., Marayong, P., Dillon, J., Chun, C.-A., and Vu, K.-P. L. (2024). “Research-infused courses are effective for online and in-person education” in Human Factors in Design, Engineering, and Computing. AHFE (2024) International Conference. AHFE Open Access. eds. T. Ahram and W. Karwowski, vol. 159 (USA: AHFE International).

Google Scholar

Balsamo, M., Lauriola, M., and Saggino, A. (2013). Work values and college major choice. Learn. Individ. Differ. 24, 110–116. doi: 10.1016/j.lindif.2012.12.022

Crossref Full Text | Google Scholar

Bastedo, M. (2021). Holistic admissions as a global phenomenon. High. Educ. Next Dec. 50, 91–114. doi: 10.1163/9789004462717_006

Crossref Full Text | Google Scholar

Carter, D. F. (2001). A dream deferred?: Examining the degree aspirations of African American and White college students. New York: Taylor & Francis.

Google Scholar

Chemers, M. M., Zurbriggen, E. L., Syed, M., Goza, B. K., and Bearman, S. (2011). The role of efficacy and identity in science career commitment among underrepresented minority students. J. Soc. Issues 67, 469–491. doi: 10.1111/j.1540-4560.2011.01710.x

Crossref Full Text | Google Scholar

Chen, G., Gully, S. M., and Eden, D. (2001). Validation of a new general self-efficacy scale. Organ. Res. Methods 4, 62–83. doi: 10.1177/109442810141004

Crossref Full Text | Google Scholar

Christopoulou, M., Lakioti, A., Pezirkianidis, C., Karakasidou, E., and Stalikas, A. (2018). The role of grit in education: a systematic review. Psychology 9, 2951–2971. doi: 10.4236/psych.2018.915171

Crossref Full Text | Google Scholar

Cole, S., and Barber, E. (2003). Increasing faculty diversity: The occupational choices of high achieving minority students. Cambridge, MA: Harvard University Press.

Google Scholar

Credé, M. (2018). What shall we do about grit? A critical review of what we know and what we don’t know. Educ. Res. 47, 606–611. doi: 10.3102/0013189X18801322

Crossref Full Text | Google Scholar

Duckworth, A. L., and Quinn, P. D. (2009). Development and validation of the short grit scale (GRIT–S). J. Pers. Assess. 91, 166–174. doi: 10.1080/00223890802634290

PubMed Abstract | Crossref Full Text | Google Scholar

Duckworth, A. L., Quinn, P. D., and Tsukayama, E. (2021). Revisiting the factor structure of grit: a commentary on Duckworth and Quinn (2009). J. Pers. Assess. 103, 573–575. doi: 10.1080/00223891.2021.1942022

PubMed Abstract | Crossref Full Text | Google Scholar

Dweck, C. S. (2006). Mindset: The new psychology of success. New York: Random House Publishing Group.

Google Scholar

Eagan, M. K., Hurtado, S., Chang, M. J., Garcia, G. A., Herrera, F. A., and Garibay, J. C. (2013). Making a difference in science education: the impact of undergraduate research programs. Am. Educ. Res. J. 50, 683–713. doi: 10.3102/0002831213482038

PubMed Abstract | Crossref Full Text | Google Scholar

Eagan, M. K., Romero, A. L., and Zhong, S. (2023). BUILDing an early advantage: an examination of the role of strategic interventions in developing first-year undergraduate students science identity. Res. High. Educ. 65, 181–207. doi: 10.1007/s11162-023-09745-8

PubMed Abstract | Crossref Full Text | Google Scholar

Estrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Gutiérrez, C. G., et al. (2016). Improving underrepresented minority student persistence in STEM. CBE Life Sci. Educ. 15:es5. doi: 10.1187/cbe.16-01-0038

PubMed Abstract | Crossref Full Text | Google Scholar

Frantz, K. J., Demetrikopoulos, M. K., Britner, S. L., Carruth, L. L., Williams, B. A., Pecore, J. L., et al. (2017). A comparison of internal dispositions and career trajectories after collaborative versus apprenticed research experiences for undergraduates. CBE Life Sci. Educ. 16:ar1. doi: 10.1187/cbe.16-06-0206

PubMed Abstract | Crossref Full Text | Google Scholar

Gilmore, J., Vieyra, M., Timmerman, B., Feldon, D., and Maher, M. (2015). The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. J. High. Educ. 86, 834–863. doi: 10.1080/00221546.2015.11777386

Crossref Full Text | Google Scholar

Guo, J., Tang, X., and Xu, K. M. (2019). Capturing the multiplicative effect of perseverance and passion: measurement issues of combining two grit facets. Proc. Natl. Acad. Sci. 116, 3938–3940. doi: 10.1073/pnas.1820125116

PubMed Abstract | Crossref Full Text | Google Scholar

Hall, A. K., Miklos, A., Oh, A., and Gaillard, S. D., (2016). Educational outcomes from the maximizing access to research careers undergraduate student training in academic research (MARC U-STAR) Program. Available at: https://www.nigms.nih.gov/News/reports/Documents/MARC-paper031416.pdf (Accessed July 10, 2022).

Google Scholar

Harpalani, V. (2023). Secret admissions. JC UL 48, 325–368.

Google Scholar

Henderson, M. C., Fancher, T. L., and Murin, S. (2023). Holistic admissions at UC Davis—journey toward equity. JAMA 330, 1037–1038. doi: 10.1001/jama.2023.15872

PubMed Abstract | Crossref Full Text | Google Scholar

Herber, S. P. (2018). The role of information in the application for highly selective scholarships: evidence from a randomized field experiment. Econ. Educ. Rev. 62, 287–301. doi: 10.1016/j.econedurev.2017.12.001

Crossref Full Text | Google Scholar

Hernandez, P. R., Woodcock, A., Estrada, M., and Schultz, P. W. (2018). Undergraduate research experiences broaden diversity in the scientific workforce. Bioscience 68, 204–211. doi: 10.1093/biosci/bix163

Crossref Full Text | Google Scholar

Hodge, B., Wright, B., and Bennett, P. (2017). The role of grit in determining engagement and academic outcomes for university students. Res. High. Educ. 59, 448–460. doi: 10.1007/s11162-017-9474-y

PubMed Abstract | Crossref Full Text | Google Scholar

Hurtado, S., Griffin, K. A., Arellano, L., and Cuellar, M. (2008). Assessing the value of climate assessments: Progress and future directions. J. Divers. High. Educ. 1, 204–221. doi: 10.1037/a0014009

PubMed Abstract | Crossref Full Text | Google Scholar

Hurtado, S., White-Lewis, D., and Norris, K. (2017). Advancing inclusive science and systemic change: the convergence of national aims and institutional goals in implementing and assessing biomedical science training. BMC Proc. 11, 17–13. doi: 10.1186/s12919-017-0086-5

PubMed Abstract | Crossref Full Text | Google Scholar

Johnson, M. K., and Elder, G. H. (2002). Educational pathways and work value trajectories. Sociol. Perspect. 45, 113–138. doi: 10.1525/sop.2002.45.2.113

Crossref Full Text | Google Scholar

Kingsford, L., Mendoza, R., Dillon, J., Chun, C.-A., and Vu, K.-P. L. (2022). Broadening and diversifying the behavioral and biomedical research workforce through early access to an undergraduate research training program. U. Int. 13, 1–24.

Google Scholar

Lewis, L., Biederman, D., Hatch, D., Li, A., Turner, K., and Molloy, M. A. (2021). Outcomes of a holistic admissions process in an accelerated baccalaureate nursing program. J. Prof. Nurs. 37, 714–720. doi: 10.1016/j.profnurs.2021.05.006

PubMed Abstract | Crossref Full Text | Google Scholar

Livinƫi, R., Gunnesch-Luca, G., and Iliescu, D. (2021). Research self-efficacy: a meta-analysis. Educ. Psychol. 56, 215–242. doi: 10.1080/00461520.2021.1886103

Crossref Full Text | Google Scholar

Mervis, J. (2003). NIH program gives minorities a chance to make their Marc. Science 301, 455–456. doi: 10.1126/science.301.5632.455

PubMed Abstract | Crossref Full Text | Google Scholar

Midkiff, B., Langer, M., Demetriou, C., and Panter, A. T. (2018). “An IRT analysis of the growth mindset scale” in Quantitative psychology. IMPS 2017. Springer proceedings in Mathematics & Statistics. eds. M. Wiberg, S. Culpepper, R. Janssen, J. González, and D. Molenaar, vol. 233. (Cham Switzerland: Springer).

Google Scholar

Miller, A. N., and Orsillo, S. M. (2020). The impact of values congruity on underrepresented minority (URM) graduate student psychosocial functioning. High. Educ. Res. Dev. 41, 450–465. doi: 10.1080/07294360.2020.1845618

PubMed Abstract | Crossref Full Text | Google Scholar

National Science Foundation (2024). Women, minorities, and persons with disabilities in science and engineering. Available at: http://www.nsf.gov/statistics/nsf03312/pdfstart.htm (Accessed February 20, 2024).

Google Scholar

NIH PAR 99-091 (1999). Minority access to research careers (MARC) ancillary training activities grants, Program Announcement, National Institute of General Medical Sciences, released on April 20, 1999.

Google Scholar

Radunzel, J., Mattern, K., and Westrick, P. (2016). The role of academic preparation and interest on STEM success. ACT research report series, 8. ACT, Inc.

Google Scholar

Ramos, H. V., Cobian, K. P., Srinivasan, J., Christie, C. A., Crespi, C. M., and Seeman, T. (2024). Investigating the relationship between participation in the Building infrastructure leading to diversity (BUILD) initiative and intent to pursue a science career: a cross-sectional analysis. Eval. Program Plann. 102:102380. doi: 10.1016/j.evalprogplan.2023.102380

PubMed Abstract | Crossref Full Text | Google Scholar

Sedlacek, W. E. (2005). “The Case for Noncognitive Measures” in Choosing students: Higher education admissions tools for the 21st century. eds. Camara W. J. and Kimmel E. W. (New Jersey: Lawrence Erlbaum Associates), 187–204.

Google Scholar

Semsar, K., Knight, J. K., Birol, G., and Smith, M. K. (2011). The Colorado learning attitudes about science survey (CLASS) for use in biology. CBE Life Sci. Educ. 10, 268–278. doi: 10.1187/cbe.10-10-0133

PubMed Abstract | Crossref Full Text | Google Scholar

Taing, A., Nguyen-Rodriguez, S. T., Rayyes, N., Marayong, P., and Buonora, P. (2022). Student perceptions of undergraduate research-infused courses. U. Int. 13, 1–21.

Google Scholar

Vu, K.-P. L., Mendoza, R., Chun, C.-A., Dillon, J., and Kingsford, L. (2023). The CSULB BUILD scholars program: a research-intensive, upper-division program to broaden and diversify the behavioral and biomedical research workforce. U. Int. 14, 1–37.

Google Scholar

Wechsler, H. S. (1984). The rationale for restriction: ethnicity and college admission in America, 1910-1980. Am. Q. 36, 643–667. doi: 10.2307/2712865

Crossref Full Text | Google Scholar

Yeager, D. S., and Dweck, C. S. (2012). Mindsets that promote resilience: when students believe that personal characteristics can be developed. Educ. Psychol. 47, 302–314. doi: 10.1080/00461520.2012.722805

Crossref Full Text | Google Scholar