BUILDing pathways to health-related research careers in biomedical and behavioral sciences: a longitudinal evaluation of postbaccalaureate outcomes using a matched comparison group

The BUilding Infrastructure Leading to Diversity (BUILD) undergraduate research training program is funded by the National Institutes of Health (NIH) to strengthen the pipeline for underrepresented students through graduate school and into health-related research careers in the biomedical and behavioral sciences. This study evaluates the impact of BUILD participation at a Minority-Serving Institution in Southern California on graduate school outcomes up to 6 years post-graduation including doctoral program enrollment and degree attainment using a quasi-experimental design. BUILD students were compared to a propensity score matched non-BUILD group using logistic regression. Results showed BUILD students enrolled in Ph.D. programs and attained Ph.Ds. at a higher rate compared to matched peers. Findings indicate BUILD met a pivotal program objective to increase doctoral degree attainment imperative for health-related research careers in biomedical and behavioral sciences. Furthermore, results support the added value of undergraduate research programs for students from underrepresented backgrounds.

Introduction

Scientists and researchers who identify as underrepresented races/ethnicities or as women often face barriers at critical transition points through the postbaccalaureate pipeline (Bennett et al., 2020a) and faculty pathway (Ransdell et al., 2021). To increase diversity of perspectives and backgrounds in the biomedical and behavioral health sciences (BBHS) research workforce and professoriate, increased diversity in doctoral degree attainment in related fields must be achieved. This means that diversity in enrollment in doctoral degree programs in these fields is paramount. Efforts to increase representation of traditionally underrepresented groups in postbaccalaureate enrollments and degree attainments in STEM fields include evidence-based interventions at the pre-doctoral level such as undergraduate research training programs (Bayliss et al., 2018; Hall et al., 2016; Vu et al., 2023). Training programs that support strengthening the pipeline to BBHS research careers for underrepresented groups have strong commitments to diversity, equity, inclusion (DEI) and social justice. In recent years, there has been an increase of anti-affirmative action movements which could have negative impacts on efforts to accelerate the enhancement of diversity of educational and employment spaces where groups have been explicitly and implicitly excluded due to racism and racist policies. Therefore, it is now more crucial than ever to demonstrate the effectiveness of undergraduate research training programs that are open to all students and include intentional components designed to attract and support students who are traditionally underrepresented in higher education and sciences.

Research on interventions to broaden the research workforce in BBHS demonstrates positive short-term outcomes (e.g., Kingsford et al., 2022). However, methodological concerns (Linn et al., 2015) and limited data on longer-term outcomes (e.g., post-baccalaureate education and career status) warrant caution in extrapolating these findings to longer-term impacts. Specifically, studies on long-term effectiveness of programs have typically relied on self-report data, used inadequate comparison groups, and/or focused on short-term (e.g., undergraduate GPA, psychosocial constructs) or broad, longer-term goals (any graduate-level acceptance or degree attainment) rather than doctoral degrees specific to the BBHS (see Whittinghill et al., 2019 for exception, although science was broadly defined as STEM and degree attainment was not included). Unfortunately, these limitations are significant because they assume that underrepresented students who receive strong training and mentorship during their undergraduate years will naturally continue to complete a doctoral program and earn a Ph.D. degree. Yet, a recent NSF report (2024) revealed that in 2023, underrepresented minority (URM) students earned 28% of bachelor's degrees in science and engineering, but only 19.4% of doctoral degrees (while 67.4% of bachelor's degrees and 77.1% of doctorates were earned by non-URM students). Moreover, Bennett et al. (2020a) reported that only 10.1% of faculty at 4-year universities are from URM backgrounds. These findings indicate that disproportionately smaller numbers of URM undergraduate students in STEM move on to doctoral programs and even fewer attain faculty positions. Additionally, Bennett et al. (2020b) found little evidence of change in race/ethnicity and gender make-up of faculty of basic science faculty across U.S. medical schools.

To examine longer-term outcomes of research training interventions and evaluate the effectiveness of an intensive undergraduate research training program, the current study employed a quasi-experimental design to examine BBHS-specific postbaccalaureate milestones including enrollment and degree attainment up to 6 years post-undergraduate graduation. The research questions for this study were addressed using a large, multi-year sample from a BUILD undergraduate research training program at California State University, Long Beach (CSULB), a diverse, urban, R2 university. Funded by the National Institutes of Health (NIH), the CSULB BUILD Program aims to strengthen the pipeline for underrepresented students through graduate school and into health-related research careers in the biomedical and behavioral sciences. This study included a comparison group, constructed with propensity score matching, to compare undergraduate students who participated in BUILD to undergraduate students who did not participate in BUILD but were similar on a variety of demographics and other characteristics.

The CSULB BUILD Program is unique as it is large in scale and aimed specifically at undergraduate research training in health-related disciplines, broadly defined across four different colleges of Engineering, Health and Human Services, Liberal Arts, and Natural Sciences and Mathematics. It is a cohort-based model, with targeted activities based on the year that students entered the program (see Figure 1). For example, the Associates program was designed as an early intervention program for sophomore-level students (see Kingsford et al., 2022 for details) with the goal of developing their interest in pursuing research at the upper division level. About 62% of Associates were accepted into one of three NIH-funded upper-division research training programs at CSULB (BUILD Scholars, MARC U^*STAR, and RISE), with 54% continuing with the BUILD Scholars program. The Scholars program, which is described in more detail below, targets juniors with four semesters left before graduation. The 1-year Fellows program targets seniors in their last undergraduate year. The application and interview processes are the same for all three BUILD programs (see Cho et al., submitted). Moreover, all three programs include a summer research training component, support for faculty-mentored research during the year, and professional development in a learning community cohort.

Figure 1

Figure 1. Structure of the BUILD program cohorts (Associates, Scholars, and Fellows) and timeline.

The Scholars Program is an intensive 2-year program. It begins with an 8-week summer program, where students meet in a learning community twice a week for a total of 6 h to engage in general research skills training (e.g., how to read a research article, developing research questions and hypothesis, and ethics training) and professional development activities (e.g., pathway to graduate school, “elevator speeches,” and poster presentations). BUILD faculty training directors lead the learning community with the support of graduate mentors. Scholars spend the rest of the week (~30–34 h) working on research projects mentored by faculty members in their discipline. The summer program culminates with a research symposium where the trainees highlight their summer accomplishments via poster presentations to their faculty mentors, lab mates, families, and friends. During the academic year of Year 1, Scholars continue to work on their research projects with their faculty mentors and participate in a learning community for 1 h per week. Trainees are encouraged to attend conferences geared toward undergraduates from underrepresented backgrounds (e.g., SACNAS or ABRCMS) in the fall. Travel support is provided by BUILD to reduce financial barriers and give students the opportunity to develop their networking skills and engage with other students, researchers, and graduate program recruiters. During Year 1, Scholars are required to apply for summer research experience programs at an R1 university. Students who are not accepted or who decide not to go to an external summer research experience program continue to work with their BUILD faculty mentors over the summer and engage in the summer learning community alongside 1st-year students (with different breakout groups). During their second academic year (Year 2), Scholars continue to work with their faculty mentors on disciplinary research, receive travel funds to present their research at a professional conference in their discipline, and are mentored as they apply to graduate school. Year 2 Scholars celebrate their conclusion of the program with a commencement ceremony. The Fellows Program is intended for seniors with prior research experience but may not have known about the BUILD Program until their senior year and did not participate in the Scholar program in the prior year. Fellows participate in the 8-week summer research training component jointly with the Year 1 Scholars during the summer and engage in similar training activities as the Year 2 Scholars during the academic year.

Throughout their entire experience in the program, BUILD trainees receive programmatic mentoring (mentoring by BUILD faculty training directors and near-peer graduate mentors, see Abeywardana et al., 2020) to increase their research skills, sense of belonging, and motivation to persist in research careers. Scholars also receive significant financial support (i.e., monthly stipend, research supply fund, and travel funds) that allows them to focus on their research and educational requirements rather than seek out external employment (see Vu et al., 2023 for more detailed information about the CSULB BUILD Scholars Program). The Associates and Fellows receive hourly pay, rather than a stipend, for their research training along with research supplies and travel funds because those programs are funded under a different NIH award mechanism.

This study aims to determine whether the CSULB BUILD Student Training Programs are effective in creating a direct pathway for students from the bachelor's to Ph.D. to maximize the potential for increasing diversity in the BBHS research workforce. The following three research questions are considered for BUILD students (defined as Associates, Scholars and Fellows who started the program during the 2015–2018 academic years after completing the summer component).

Question 1: Are BUILD students enrolled in doctoral programs at a higher rate than their matched non-BUILD peers after graduating from CSULB? Are BUILD students enrolled in fields associated with the BBHS research career?

Question 2: Do BUILD students earn doctoral degrees at a higher rate than their matched non-BUILD peers? Are BUILD students' doctoral degree types (Ph.D. vs. professional doctoral degrees) different compared to those of their matched peers? Are BUILD students earning degrees in fields associated with the BBHS research career?

Question 3: Are there differences in BUILD doctoral degree outcomes by identities (i.e., gender, and race/ethnicity) and family educational status (i.e., first-generation) compared to those of the matched non-BUILD peers?

Materials and methods

Participants

The pre-matched sample pool included 281 BUILD students (comprised of Associates, Scholars, and Fellows) and 6,310 non-BUILD students at CSULB. See Table 1 for demographic characteristics of the two student groups before matching. BUILD students were from four cohorts (2015–2018) and entered the program as sophomores (Associates, 47.2%), juniors (Scholars, 50.2%), or seniors (Fellows, 2.6%).

Table 1

Table 1. Demographics of BUILD (2015–2018 cohorts) and non-BUILD students before matching.

Overview of matching process

Propensity score matching (PSM, Bai and Clark, 2019; Rosenbaum and Rubin, 1985) method was used to create a matched comparison group of non-BUILD to BUILD students. Propensity scores are predicted probabilities for selection into BUILD using predictors as matching variables in a statistical model. A PSM approach is efficient when it is cumbersome to match participants on many variables at the same time. Conceptually, matches of non-BUILD students are selected that have the same or very similar propensity scores as BUILD students. To accomplish the matching process, we used predictor variables that related to the selection into the BUILD training program. After the matching procedure and omitting data of non-matched students, statistical analyses were performed to address the research questions comparing the BUILD group to the matched non-BUILD group on outcomes such as doctoral program enrollment and degree attainment as described below.

Covariate selection

A pool of covariates was constructed from programmatic selection criteria of BUILD. In addition to all demographic variables listed in Table 2, the GPA before BUILD participation and participation in independent research were also included as potential covariates. All data were retrieved from the Office of Institutional Research and Analytics at CSULB. Next, statistical relationships were assessed between each covariate and BUILD participation. Associations were also assessed between each covariate and the last recorded GPA and graduation status. Final decisions about which covariates were included in the PSM model were based on effect sizes of t-tests, chi-square, or correlation tests (as appropriate for the levels of measurement). Cohen's d > 0.10 or r > 0.10 indicated that a covariate was not balanced between BUILD and non-BUILD groups and therefore should be included in the PSM model. For statistical tests between the variables GPA and graduation status and potential covariate, a d > 0.10 or r > 0.10 indicated a covariate was sufficiently related to the outcome to also be considered for inclusion in the PSM model.

Table 2

Table 2. Relationships between potential covariates, BUILD participation, and academic outcomes pre-matching and post-matching.

Pre-matching results are displayed in Table 2, Column 2. Several variables differed by BUILD participation status, including age, year of entry (CSULB matriculation), full-time status, transfer status, college, GPA at entry to CSULB and independent research experience, and were therefore included in the PSM model. Although other variables appeared balanced between BUILD and non-BUILD groups, results showed that these variables were related to GPA and/or graduation status (see Columns 3 and 4 in Table 2) and consequently could influence the way BUILD participation impacts graduate school outcomes. Therefore, these variables were also included in the PSM (see for example, gender). For variables that were initially balanced between BUILD and non-BUILD groups, checks were done post-matching to confirm that balance was maintained.

Estimation of propensity scores

Nearest neighbor matching (1:1) was selected because it is an efficient and commonly used method based on the closest absolute distance between propensity scores. Caliper matching was applied as it is suited for a large comparison group. Following the formula by Rosenbaum and Rubin (1985), a caliper width of 0.035 was calculated based on c = 0.25 (SD[p(x)]), where SD = standard deviation of the propensity scores (here, 0.14) to remove at least 90% of bias (Bai, 2011; Cochran and Rubin, 1973). Due to good common support (that is, non-BUILD group propensity scores overlap with most of the BUILD group) and given that the non-BUILD group was more than three times the size of the BUILD group, replacement was not needed. First, data were prepared for logistic regression. Variables that were categorical with more than two categories were dummy coded. When categories had a frequency of zero for BUILD and very small frequency for non-BUILD (< 5%), non-BUILD participants from these categories were excluded to avoid estimation problems. This included Native American and Pacific Islander categories for race/ethnicity and the College of Business Administration and College of Education categories. White was the referent group for race/ethnicity and College of Liberal Arts was the referent group for college. Collinearity was also checked between predictors because perfect collinearity will lead to non-convergence and high collinearity could result in “bouncing betas” including large standard errors, and implausible estimates. Tolerance and VIF values indicated no issues with perfect or high multicollinearity. A logistic regression with BUILD status as the outcome was estimated to examine the overall model fit, (χ² [21, N = 4,820] = 988.26, p < 0.001, Nagelkerke R² = 0.52). The model converged in nine iterations. Estimates are presented in Table 3.

Table 3

Table 3. Logistic regression predicting BUILD participation.

Evaluation of matching

Covariates could remain unbalanced after matching, and matching could potentially increase imbalance. Therefore, each covariate was examined for balance, even those that appeared balanced prior to matching. Matching for referent groups for categorical variables was also checked. Balance was assessed by examining the magnitude of difference for each covariate between BUILD and non-BUILD groups. Standardized mean differences were calculated (SMD, based on Cohen's d). Standardized bias was calculated by multiplying the SMD by 100. A standardized bias < 10% reflects adequate balance (coinciding with Cohen's d < 0.10; Bai and Clark, 2019). These estimates appear in Table 2, Column 5. All SMDs are lower than the recommended threshold of 0.10, indicating that standardized bias is < 10% for each covariate and all relationships are non-significant. Given the level of balance achieved, the PSM model was adequate and was used to examine possible differences in outcomes between BUILD and their matched non-BUILD students. The same matched groups were used to address each research question in this study. Simulation evidence supports the use of a single, generic-outcome propensity score model when multiple, related outcomes are examined (Wyss et al., 2013).

Measures

Outcome data were retrieved from the National Student Clearinghouse (NSC) in late September 2024 to track student graduate outcomes. Variables included graduate program enrollment, enrollment major and/or the Classification of Instructional Programs (CIP; a major code), degree attainment, degree major and/or CIP codes, and type of degree attained (e.g., Ph.D., Professional doctoral degrees such as Doctor of Physical Therapy, Optometry, or Pharmacy). All variables were binary coded (e.g., 1 = enrolled, 0 = not enrolled; 1 = Ph.D. earned, 0 = No Ph.D. earned). BBHS-related majors were coded 1 (e.g., chemistry, neuropsychology), while non-BBHS related majors were coded 0 (e.g., geography, English). Classification decisions were reviewed by two independent reviewers. Regarding enrolled Ph.D. program disciplines, CIP codes and/or majors were reported for 77% of students enrolled. Major information was missing at a slightly higher rate for the non-BUILD students than for the BUILD students (29% vs. 21% missing, respectively). When all missing majors were imputed based on other information provided (e.g., type of doctorate or master's degree earned, and their major in another program), no substantive differences in results were found. Therefore, we used the imputed CIP/major codes for completeness. Majors were further categorized as either biomedical (i.e., natural sciences, specific engineering type majors such as biomedical engineering, chemistry, coded 1) or behavioral disciplines (i.e., liberal arts, specific health and human services type majors such as psychology or physical therapy, coded 0). BUILD status was binary coded (i.e., 1 = BUILD student, 0 = non-BUILD student).

Data analysis

IBM SPSS (v.29; IBM Corp, 2023) was used for all analyses. Binary logistic regression models were estimated due to binary outcome variables. Some student records were not obtained. Per NSC, data are missing when students have their records blocked or partially blocked, when student records were not found, or when an institution does not report data to NSC. With < 7% of data missing, this reduced the total sample size to 495 (N_BUILD = 268, N_{Non − BUILD} = 227). We reexamined the evaluation of matched groups based on this reduced sample size due to missing data. The analysis reflected that missingness did not have an impact on the matching of the groups, and thus, groups were still well matched (see Table 2). Odds ratios (OR), a natural effect size, were reported and interpreted when the predictor was statistically significant. The Nagelkerke R-squared (R²_N) was also reported as an effect size. The R²_N was selected as a pseudo-R² because it ranges from 0 to 1, with higher values indicating larger effect sizes. If the expected count for a category was very small (e.g., < 5), a Fisher's Exact test was analyzed instead. All inferential tests were two-tailed tests (α = 0.05).

Results

Research question 1

Are BUILD students enrolled in doctoral programs at a higher rate than their matched non-BUILD peers after graduating from CSULB? Are BUILD students enrolled in fields associated with the BBHS research career?

Counts of doctoral enrollment by BUILD program participation status are displayed in Table 4. Overall, there were 128 total enrollments (25.9%) in doctoral programs (86 BUILD student enrollments and 42 non-BUILD student enrollments). There was a statistically significant difference in the number of doctoral enrollments (regardless of type) by BUILD status, X²(1) = 12.06, p < 0.001, $R_{\mathit{\text{N}}}^2$ = 0.04. The odds of enrolling in any doctoral degree program for BUILD students were 2.08 times greater than the odds for non-BUILD matched students, b = 0.73, SE = 0.22, OR = 2.08, p < 0.001. Moreover, 88 students (17.8% of all students) were enrolled in a Ph.D. granting graduate program. A little more than a quarter (26.5%, N = 71) of BUILD students and 7.5% (N = 17) of non-BUILD students were enrolled. The difference in enrollment in Ph.D. program between the two groups was statistically significant, X²(1) = 32.63, p < 0.001, $R_{\text{N}}^2$ = 0.11. For BUILD students the odds of enrollment in a Ph.D. program were 4.45 times the odds for non-BUILD students, b = 1.49, SE = 0.29, OR = 4.45, p < 0.001. Regarding the disciplines, a slightly higher proportion of BUILD students enrolled in BBHS-related disciplines than non-BUILD students (93.0%, N = 66; 82.4%, N = 14, respectively). However, a Fisher's Exact test indicated a non-significant association between BUILD status and the BBHS-related discipline of Ph.D. program enrolled, p = 0.180. Within the BBHS-related disciplines, 71.2% (N = 47) of the BUILD Ph.D. enrollments were in biomedical disciplines while 28.8% (N = 19) were in behavioral disciplines. For the 14 non-BUILD students enrolled in Ph.D. programs within the BBHS-disciplines, 85.7% (N = 12) were in biomedical disciplines while 14.3% (N = 2) were in behavioral. A Fisher's Exact test indicated that Ph.D. enrollment discipline types for BBHS enrollments did not differ between BUILD and non-BUILD students, p = 0.334.

Table 4

Table 4. Counts and odds of doctoral enrollment by BUILD status and type of degree.

A small percentage of students were enrolled in professional degree granting programs (7.9%, N = 40) with a higher percentage of non-BUILD students (11.1%, N = 25) attending professional programs than BUILD students (5.6%, N = 15). There was a significant difference in professional degree program enrollment by BUILD status, X²(1) = 4.86, p = 0.028, $R_{\mathit{\text{N}}}^2$ = 0.02. The odds of BUILD students enrolling in a professional degree program were about half the odds of non-BUILD students (b = −0.74, SE = 0.34, p = 0.030, OR = 0.48). Of the 15 BUILD professional degree program enrollments, 93.3% (N = 14) were in BBHS-related majors. Of the 25 enrollments for non-BUILD students, 88.0% (N = 22) were in BBHS-related majors. A Fisher's Exact test indicated a non-significant association between BUILD status and enrollment BBHS-related major for the professional degree program enrollments, p = 0.999.

Research question 2

Do BUILD students earn doctoral degrees at a higher rate than their matched non-BUILD peers? Are BUILD students' doctoral degree types (Ph.D. vs. professional doctoral degrees) different compared to those of their matched peers? Are BUILD students earning degrees in fields associated with the BBHS research career?

Counts of degrees attained by BUILD program participation status are displayed in Table 5. Overall, 42 doctorate degrees were earned by late September 2024 (20 Ph.Ds. and 22 professional degrees). BUILD students earned a total of 30 doctorate degrees (18 Ph.Ds. and 12 professional degrees) while non-BUILD students earned 12 doctorate degrees (2 Ph.Ds. and 10 professional degrees). Logistic regression results indicated that there was a statistically significant difference in number of doctorate degrees (regardless of type) earned by BUILD participation status, X²(1) = 5.74, p = 0.017, $R_{\text{N}}^2$ = 0.03. The odds of earning any doctoral degree for BUILD students were 2.26 times greater than the odds for matched non-BUILD students, b = 0.82, SE = 0.35, OR = 2.26, p = 0.021. Additionally, there was a statistically significant difference in Ph.D. attainment by BUILD status, X²(1) = 12.64, p < 0.001, R²_N = 0.09. The odds of earning a Ph.D. were 8.10 times for BUILD students compared to matched non-BUILD students, b = 2.09, SE = 0.75, OR = 8.10, p = 0.005. Lastly, there was not a statistically significant difference in professional degree attainment by BUILD status, X²(1) = 0.00, p = 0.969. BUILD students were equally likely to earn a professional degree as matched non-BUILD students, b = 0.02, SE = 0.44, OR = 1.02, p = 0.969. Regarding discipline, only two degrees were earned in non-BBHS disciplines. Both were professional degrees, and one was earned by a BUILD student and the other, a non-BUILD student.

Table 5

Table 5. Counts and odds of doctoral degrees by BUILD status and type of degree.

Research question 3

Are there differences in BUILD doctoral degree outcomes by identities and family educational status compared to those of the matched non-BUILD peers?

Descriptive statistics summarizing demographics for trainees enrolled in a doctorate program are displayed in Figure 2. Of the 128 doctorate degree program enrollees, a majority were women (60.2%), over half were awarded a Pell-grant as an undergraduate (55.5%), a fifth were first-generation students (20.3%), and 14.8% were undergraduate transfer students. Regarding race/ethnicity, the two largest groups identified as Hispanic/Latinx (42.2%) and Asian American (33.6%). See Figure 2 for demographics for doctoral degree program enrollment disaggregated by BUILD participation status. We caution not to overinterpret differences of percentages, particularly for very small group sizes. Therefore, counts are also reported in parentheses to supplement the percentages. Since some categories had small proportions and/or the group size proportions differ largely between BUILD and non-BUILD groups for some demographics, the power to detect significant differences in enrolling in a doctoral degree program between demographic groups by BUILD status was quite limited. Therefore, inferential tests of interactions were not used to determine if demographic differences of educational outcomes between BUILD and non-BUILD were statistically significant.

Figure 2

Figure 2. Demographics for students enrolled in doctoral degree programs by BUILD status.

When analyzing data by demographic subsamples, logistic regression results in Table 6 revealed that female BUILD students (p < 0.001), first-generation BUILD students (p = 0.001), Pell-eligible BUILD students (p = 0.010), transfer BUILD students (p = 0.023) and URM BUILD students (p = 0.001) all enrolled in more doctoral programs than their respective non-BUILD counterparts. Notably, the largest effect size was for first generation students ($R_N^2$ = 0.12). The odds of enrolling in a doctoral program for first generation BUILD students were 4.79 times the odds for first generation non-BUILD students. Demographic information for Ph.D. and professional degree program enrollments is provided in supplemental materials (Tables 7, 8) as results are comparable to the overall results.

Table 6

Table 6. Logistic regression model results for doctoral program enrollment regressed on BUILD status by subsample.

Table 7

Table 7. Student demographics for enrolled in professional program by BUILD status.

Table 8

Table 8. Student demographics for enrolled in PhD program by BUILD status.

Descriptive statistics summarizing demographics for doctorate earners are displayed in Figure 3. Of the 42 doctorate degree earners, a majority were women (66.7%), over half were awarded a Pell-grant as an undergraduate (57.1%), about a quarter were first-generation students (23.8%), and about a fifth were undergraduate transfer students (21.4%). Regarding race/ethnicity, again the two largest groups identified as Asian American (40.5%) and Hispanic/Latinx (31.0%). See Figure 3 demographics for doctorate degree earners disaggregated by BUILD participation status. Since the overall group size of doctorate degree earners was small and reasons indicated above, inferential tests of interactions were not used to determine if demographic differences of educational outcomes between BUILD and non-BUILD were statistically significant. When examining subsamples (see Table 9), female BUILD students (N = 21, 13%) earned more doctoral degrees than female non-BUILD students (N = 7, 4.3%), X²(1) = 5.76, p = 0.016, $R_N^2$= 0.04. The odds of earning a doctoral degree for female BUILD students were 2.79 times the odds for female non-BUILD students, b = 1.03, SE = 0.45, OR = 2.79, p = 0.024. Additionally, first generation BUILD students earned more doctoral degrees (N = 9, 12.7%) than first generation non-BUILD students (N = 1, 1.6%), X²(1) = 6.76, p = 0.009, $R_N^2$= 0.12. The odds of earning a doctoral degree for first generation BUILD students were 8.86 times the odds for first generation non-BUILD students, b = 2.18, SE = 1.07, OR = 8.86, p = 0.041. There was not a statistically significant difference between BUILD and non-BUILD students in their doctoral degree attainment for any other demographic category (see Table 9 for full results).

Figure 3

Figure 3. Student demographics for doctorate degree earners by BUILD status.

Table 9

Table 9. Logistic regression model results for doctoral degree attainment regressed on BUILD status by subsample.

Discussion

This study aimed to assess the effectiveness of the CSULB BUILD Student Training Programs in providing a direct pipeline to Ph.D. programs. Although other studies have investigated the CSULB BUILD Scholars Program's outcomes using a propensity score matching approach (Ramos and Vu, 2024; Stormes et al., 2022), this study is the first to examine the longer-term program outcomes of Ph.D. program enrollment and degree attainment with NSC data. Utilizing a quasi-experimental design, the results indicated that BUILD trainees enrolled in doctoral programs at significantly higher rates than their matched non-BUILD peers (26.9% vs. 7.9%). Furthermore, most of the BUILD trainees (93%) pursued doctoral degrees in BBHS-related fields, underscoring the program's success in aligning with its intended goals. Notably, the odds of earning a Ph.D. were 8.10 times for BUILD students compared to matched non-BUILD students, although there was no significant difference in the attainment of professional degrees. This result likely reflects BUILD/NIH's programmatic goals including BUILD Programs' recruitment/selection process that emphasizes Ph.D. degree attainment and actively screens out those who may be solely interested in pursuing professional degrees (e.g., medicine). Still, BUILD students may not disclose their interests in professional degree programs or may not be fully committed to their stated graduate degree goals, especially early on. Lastly, regarding race/ethnicity, enrollment to doctoral programs was greater for BUILD URM students compared to non-BUILD URM students. Although small sample sizes limited the ability to perform interactions and statistical comparisons between all demographic groups, disaggregated data suggest that BUILD students from underrepresented groups enrolled in doctoral programs at higher rates than non-BUILD students including women, first generation, Pell eligible, and transfer students. Regarding doctoral attainment, BUILD had more women and first-generation doctorate earners compared to their non-BUILD counterparts. Furthermore, the largest proportion of degree earners were Asian American and Latine for both BUILD and non-BUILD groups.

Findings from the present study align with what past research has shown on undergraduate research training programs' effectiveness with supporting the progression of underrepresented groups into STEM research careers (Bayliss et al., 2018; Hall et al., 2016; Vu et al., 2023). Moreover, they provide direct and highly encouraging evidence for the longer-term outcomes that most past research could only suggest or imply. Specifically, the higher doctoral program enrollment and degree attainment rates in BBHS-related disciplines among BUILD students found in this study demonstrate the efficacy of these interventions as a viable pathway for students entering research careers, including for those from historically underrepresented and less resourced groups.

Several limitations should be considered when interpreting these findings. The small sample size of total doctorate earners may limit the generalizability of the results. This study used a variety of covariates (demographic and academic, including research experience) to create the comparison group. However, the quasi-experimental design, while robust, cannot fully eliminate selection bias like randomization. The use of propensity score matching, though effective in creating comparable groups, may not account for all unobserved variables influencing doctoral degree attainment. Moreover, due to the fact that multiple cohorts are included, the timeframe of the data collection gives more time for doctoral program enrollment and degree attainment for students from earlier cohorts. It is also important to note that enrolled students in doctoral programs are continuing their progress toward doctoral attainment. This likely underestimation of the longer-term outcomes suggests that our current results may be a conservative test of our hypotheses. Currently, more BUILD students are enrolled in Ph.D. programs than non-BUILD students (and vice versa for professional degree program enrollments). Therefore, if students persist to completion, there is potential for more of a pronounced difference in type of degree attainment between BUILD and non-BUILD groups as well as other group differences (e.g., race/ethnicity). Both BUILD and non-BUILD students may also continue to enroll in doctoral programs and may also graduate in the future. This may be especially true for disciplines where prospective students are commonly expected to obtain a master's degree prior to admission to Ph.D. programs. Furthermore, demographic representation of doctoral degree by gender and race/ethnicity largely depends on the discipline of degree. As additional degrees are earned and the sample size grows, future research could turn to examining demographic representation by types of disciplines of degrees. Other outcomes of interest beyond doctoral outcomes could also be examined including outcomes related to enrollment (e.g., applications to graduate school), graduate outcomes like fellowships/awards and publications, and longer-term outcomes like career attainment, though data on these outcomes may be more difficult and costly to collect longer-term.

The present study's findings can guide institutional priorities and funder efforts aimed at implementing effective practices. That is, our analyses show the value of tracking students in the long term, which requires resources. Having funding for such efforts would allow future research to explore long-term career outcomes of student research training programs, such as the BUILD Program, beyond doctoral degree attainment. These data will allow researchers to draw stronger conclusions about whether such programs do indeed lead to a more diverse BBHS research workforce and which specific program components, such as mentoring, learning communities, and summer research experiences, shape the long-term outcomes.

In conclusion, the CSULB BUILD Student Training Programs have demonstrated significant potential in strengthening the pipeline for Ph.D. attainment as well as fostering diversity within the BBHS research workforce by increasing the representation of underrepresented groups in Ph.D. programs and among Ph.D. degree earners. This study highlights the importance of sustained support for such initiatives to ensure the continued advancement of diversity, equity, and inclusion in the biomedical and behavioral health sciences.

Data availability statement

The data analyzed in this study is subject to the following licenses/restrictions: The dataset includes sensitive information including school records over multiple institutions that could identify the subjects of this research study, especially for individuals from underrepresented minority groups given the specificity of the information including demographics and school record information even when the data are scrubbed of names and ID numbers. Requests to access these datasets should be directed to Erin H. Arruda at RXJpbi5BcnJ1ZGFAY3N1bGIuZWR1.

Ethics statement

The studies involving humans were approved by California State University, Long Beach IRB. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

EA: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Writing – original draft, Writing – review & editing. K-PV: Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Resources, Supervision, Validation, Writing – original draft, Writing – review & editing. C-AC: Conceptualization, Data curation, Funding acquisition, Methodology, Project administration, Writing – original draft, Writing – review & editing. GG: Writing – original draft, Writing – review & editing. PM: Data curation, Funding acquisition, Project administration, Writing – original draft, Writing – review & editing. JD: Data curation, Funding acquisition, Project administration, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was supported by the National Institute of General Medical Sciences of the National Institutes of Health Awards UL1GM118979, TL4GM118980, and RL5GM118978.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

Abeywardana, S. U., Velasco, S., Hall, N., Dillon, J., and Chun, C.-A. (2020). Near-peer mentoring in an undergraduate research training program at a large master's comprehensive institution. Understand. Interven. 11:12477.

PubMed Abstract | Google Scholar

Bai, H. (2011). Using propensity score analysis for making causal claims in research articles. Educ. Psychol. Rev. 23, 273–278. doi: 10.1007/s10648-011-9164-9

Crossref Full Text | Google Scholar

Bai, H., and Clark, M. H. (2019). Propensity Score Methods and Applications. Thousand Oaks, CA: Sage. doi: 10.4135/9781071814253

Crossref Full Text | Google Scholar

Bayliss, F., Peterfreund, A., and Rath, K. (2018). “Programmatic mentoring,” in Mentoring at Minority Serving Institutions (MSIs): Theory, Design, Practice and Impact (eds) J. McClinton, D. S. Mitchell, G. B. Hughes, and M. A. Melton (Charlotte, NC: Information Age Publishing, Inc.).

Google Scholar

Bennett, C. L., Salinas, R. Y., Locascio, J. J., and Boyer, E. W. (2020b). Two decades of little change: an analysis of US medical school basic science faculty by sex, race/ethnicity, and academic rank. PLoS ONE 15:e0235190. doi: 10.1371/journal.pone.0235190

PubMed Abstract | Crossref Full Text | Google Scholar

Bennett, J., Lattuca, L., Redd, K., and York, T. (2020a). Strengthening Pathways to Faculty Careers in STEM: Recommendations for Systemic Change to Support Underrepresented Groups. Lessons from the APLU INCLUDES Project. Association of Public and Land-grant Universities. Available at: https://www.aplu.org/wp-content/uploads/strengthening-pathways-to-faculty-careers-in-stem-recommendations-for-systemic-change-to-support-underrepresented-groups.pdf.

Google Scholar

Cochran, W. G., and Rubin, D. B. (1973). Controlling bias in observational studies: a review. Sankhyā Ind. J. Statistics Series A 35, 417–446.

Google Scholar

Hall, A. K., Miklos, A., Oh, A., and Gaillard, S. D. (2016). Educational Outcomes from the Maximizing Access to Research Careers Undergraduate Student Training in Academic Research (MARC U-STAR) Program. Available at: https://www.nigms.nih.gov/about/dima/Documents/MARC-paper031416.pdf (accessed 10 July 2022).

Google Scholar

IBM Corp (2023). IBM SPSS Statistics for Windows, Version 29.0.2.0. Armonk, NY: IBM Corp

Google Scholar

Kingsford, L., Mendoza, R., Dillon, J., Chun, C.-A., and Vu, K.-P. (2022). Broadening and diversifying the behavioral and biomedical research workforce through early access to an undergraduate research training program. Understand. Interven. 13, 1–24.

PubMed Abstract | Google Scholar

Linn, M. C., Palmer, E., Baranger, A., Gerard, E., and Stone, E. (2015). Undergraduate research experiences: impacts and opportunities. Science 347, 627–633. doi: 10.1126/science.1261757

PubMed Abstract | Crossref Full Text | Google Scholar

Ramos, H. V., and Vu, K.-P. L. (2024). Research, science identity, and intent to pursue a science career: a BUILD intervention evaluation at CSULB. Educ. Sci. 14:647. doi: 10.3390/educsci14060647

PubMed Abstract | Crossref Full Text | Google Scholar

Ransdell, L. B., Lane, T. S., Schwartz, A. L., Wayment, H. A., and Baldwin, J. A. (2021). Mentoring new and early-stage investigators and underrepresented minority faculty for research success in health-related fields: An integrative literature review (2010–2020). Int. J. Environ. Res. Public Health 18:432. doi: 10.3390/ijerph18020432

PubMed Abstract | Crossref Full Text | Google Scholar

Rosenbaum, P. R., and Rubin, D. B. (1985). Constructing a control group using multivariate matched sampling methods that incorporate the propensity score. Am. Stat. 39, 33–38. doi: 10.1080/00031305.1985.10479383

Crossref Full Text | Google Scholar

Stormes, K., Streicker, N. A., Bowers, G. K., Ayala, P., and Urizar, G. (2022). Impact of undergraduate research training programs: an illustrative example of finding a comparison group and evaluating academic and graduate school outcomes. Assessment 5, 25–36. doi: 10.18833/spur/5/3/8

PubMed Abstract | Crossref Full Text | Google Scholar

Vu, K.-P., Mendoza, R., Chun, C.-A., Dillon, J., and Kingsford, L. (2023). The CSULB BUILD Scholars program: a research intensive upper-division program to broaden and diversify the behavioral and biomedical research workforce. Understand. Interven. 14, 1–37.

PubMed Abstract | Google Scholar

Whittinghill, J. C., Slovacek, S. P., Flenoury, L. P., and Miu, V. (2019). A 10-year study on the efficacy of biomedical research support programs at a public university. Scholarship Pract. Undergrad. Res. 3, 30–38. doi: 10.18833/spur/3/1/3

Crossref Full Text | Google Scholar

Wyss, C. J., LoCasale, R. J., Brookhart, A. M., and Stürmer, T. (2013). Variable selection for propensity score models when estimating treatment effects on multiple outcomes: a simulation study. Pharmacoepidemiol. Drug Saf. 22, 77–85. doi: 10.1002/pds.3356

PubMed Abstract | Crossref Full Text | Google Scholar