NHERI education and community outreach: longitudinal, educational outcomes of a coordinated, distributed network

The U.S. National Science Foundation (NSF) Natural Hazards Engineering Research Infrastructure (NHERI) Network Coordination Office (NCO) Education and Community Outreach (ECO) led coordinated efforts to promote educational activities along various pathways for students and educators targeted at broadening participation in and awareness of natural hazards engineering research through the Research Experiences for Undergraduates (REU) Summer Program, the Graduate Student Council (GSC), and the Summer Institute for Early Career Researchers and K-12 Educators. NHERI connects a diverse group of undergraduate and graduate students, faculty and K-12 educators, and researchers interested in mitigating the effects of natural hazards through these flagship educational programs. After 6 years of implementing these integrated educational activities, longitudinal outcomes and impacts for both students and faculty have been collected and are reported in this paper. Embedded in this report are several best practices used in educational outreach for recruitment, mentoring, and engagement of diverse participants that have been evaluated and enhanced through assessment and in collaboration with the larger NHERI network. Throughout 6 years of leading education activities, these practices have also helped create an intentional focus on challenge areas and informed the evolution of interdisciplinary pathways for natural hazards engineering research.

1 Introduction

As a diverse science, technology, engineering, and math (STEM) workforce continues to be a critical area for improvement within the United States (National Academies of Science, Engineering and Medicine, 2016), the U.S. National Science Foundation (NSF) Natural Hazards Engineering Research Infrastructure (NHERI) Network Communication Office (NCO) Education Community Outreach (ECO), or NHERI ECO for short, developed inclusive pathways to support members of underrepresented groups (URGs) in pursuit of STEM careers. These URGs include women and historically marginalized racial and ethnic participants (American Society for Engineering Education, 2022). As part of a large, distributed network of natural hazards engineering research and experimental facilities across twelve universities and thirteen components, the NHERI ECO developed educational programming to prepare and support future natural hazards engineers and researchers. These programs are developed, executed, and assessed in collaboration with these network components. In collaboration with educational experts, educational activities are also designed using evidence-based practices. These practices are adapted to focus on engaging the various targeted audiences that the NHERI ECO aims to reach. This paper presents the NHERI ECO programming as a whole and its impact on broadening participation for URG within the STEM workforce and specifically, the natural hazards engineering research community.

2 NSF NHERI NCO ECO programs

The NHERI ECO programs are organized and orchestrated collaboratively between the NHERI ECO and the ECO Committee. The ECO Committee is composed of representatives from each of the NHERI components (See Figure 1 for details) and is chaired by a member of the NHERI ECO responsible for the education activities. The ECO Committee meets monthly to strategically organize network-wide education and outreach activities. NHERI facility representatives who serve on the ECO Committee also play an essential role in communicating between their sites, other NHERI facilities, and the NCO. Working together, NHERI ECO launched two flagship educational programs in 2017, the NSF NHERI Research Experiences for Undergraduates (REU) Summer Program and the NSF NHERI Summer Institute for Early-Career Researchers and K-12 Educators. Both programs were developed to illuminate pathways for members of underrepresented groups (URGs) in STEM and provide necessary training, career development, resource awareness, and support. In 2020, educational programs were adapted to regulations resulting from the COVID-19 pandemic. In 2021, the virtual graduate student program, known as the NSF NHERI Graduate Student Council (GSC), was added in response to community input to create a continuous STEM workforce pathway to natural hazards engineering and research linking the two previous programs.

Figure 1

Figure 1. Diagram of NSF NHERI distributed network.

2.1 NSF NHERI REU summer program

The REU Summer Program is a hybrid (virtual and in-person) program and coordinated effort across the NHERI network sites that engages up to thirty-three, funded undergraduate students of all classes (freshman-senior) in natural hazards engineering research. Undergraduate researchers often participate in their first hands-on research project during the summer program and experience a research-based curriculum designed to introduce students to the rhetoric of scholarly writing in their specific disciplines through a scaffolded weekly approach tailored to their experience. Over the ten-week summer program, participants are mentored in person by experienced NHERI faculty and staff as they conduct research at one of the NHERI research and experimental facilities. Students are also guided and purposefully mentored by the NHERI ECO Program Coordinator and Education Specialist in weekly virtual research meetings and personal check-ins as they create their research posters, presentations, and papers. In weekly virtual meetings, they also build community with their peers across the NHERI sites by sharing their experiences and problem-solving challenges. The program ends with an in-person culminating event, the NSF NHERI REU Symposium, hosted at one of the NHERI components that provides participants an opportunity to share research and network with their peers. The REU program offers research and career training, a network of mentors, and socializing in the engineering and research community that prepares participants for future careers in STEM, specifically in natural hazard mitigation.

2.2 NSF NHERI graduate student council

In response to an observed need to bridge the undergraduate experience with the needs of early career faculty, NHERI ECO also developed a program that supports graduate students through the GSC which was launched in October of 2021 as a student-led virtual organization. Designed to connect like-minded graduate students from around the world, the GSC invites graduate students interested in mitigating natural hazards to connect with peers and prominent researchers from NHERI and beyond. The group is committed to offering learning opportunities and career development programs to graduate students without registration fees. Beyond building community, the organization offers graduate students leadership opportunities, workshops, mini-conferences, general monthly meetings with speakers from the natural hazard community, and various funding opportunities. GSC members also mentor REU students, serve on panels, and lead presentations that benefit and engage undergraduate researchers interested in natural hazards engineering. The GSC has become an integral piece of the education pathways within NHERI, supporting and preparing the next-generation of the STEM workforce.

2.3 NSF NHERI summer institute for early career faculty

Finally, the NHERI ECO engages early-career faculty and researchers in preparation to successfully begin and continue their academic and research careers. The Summer Institute for Early Career Researchers is a three-day intensive workshop focused on introducing NSF grant writing, presenting NHERI resources (i.e., experimental facilities, NSF NHERI Science Plan, NSF NHERI Technology Transfer Committee, DEI/Broader Impacts), and building community through networking that supports twenty funded early career faculty, five GSC members, and five K-12 certified teachers as education consultants. NHERI faculty and staff from all eleven sites present information on experimental, simulation, reconnaissance, and cyberinfrastructure resources. Site representatives also mentor participants and help scaffold a grant writing experience with a series of workshops, speakers, and panels. The event ends with presentations from collaborative groups of early career faculty. They present a mock grant proposal created during the workshop to a panel of judges selected from the natural hazards engineering community. A top proposal is selected as the winning team, and all interdisciplinary groups receive proposal feedback from accomplished NSF-funded researchers. This unique workshop offers information about NHERI experimental facilities, components, resources, and awareness to begin the proposal process in partnership with a NHERI site using the Science Plan, knowledge about NSF grant guidelines, and guidance from a Civil, Mechanical, and Manufacturing Innovation (CMMI) NSF Director, practice detailing broader impacts in an education plan with K-12 education consultants, and the opportunity to actively use this information and mentorship with peers to create a mock grant proposal.

3 Methodology

Using a longitudinal case study design (Singer and Willet, 2003; Yin, 2009), the NHERI ECO collected demographic application/registration data for each program, tracked longitudinal impact data, and analyzed changes in pre-program and post-program assessment surveys for participants funded through the REU and Summer Institute. All pre- and post-assessments used a Likert scale ranging from one (1), “Strongly disagree/Not at all,” to five (5) “Strongly agree/A great deal.”

The demographic information collected for each program included race/ethnicity, gender, first-generation college student status, home university’s Carnegie classification (American Council on Education, 2024), geographic location, research experience, and major/career focus. Demographic and diversity data were collected for all three programs as seen in Table 1 (REU), Table 2 (NHERI GSC), and Table 3 (Summer Institute). Additionally, the demographic data were compared to national data from the American Society of Engineering Education (2022) to place the program in a national civil engineering context.

Table 1

Table 1. NSF NHERI REU demographic breakdown ^a2017–2023.

Table 2

Table 2. NSF NHERI GSC demographic breakdown^a 2021–2023.

Table 3

Table 3. NSF NHERI Summer Institute demographic breakdown^a 2017–2023.

Further, longitudinal data were collected for the REU and Summer Institute participants through LinkedIn, Google Scholar, and/or NSF Award Search to track career progress after participation. This data specifically identified graduation date, discipline and level of degree attainment, and current employment status for past participants of both programs as well as the number of publications and amount of NSF funding awarded to Summer Institute alumni. In the future, NHERI GSC longitudinal data will be collected via a member graduate exit survey as well as with participant surveys.

The pre-program and post-program self-assessment data were collected for both REU and Summer Institute participants and analyzed using a paired sample t-test for its robustness against error to determine changes in research self-efficacy in participants using mean-difference (Fradette, et al., 2003; Rasch and Guiard, 2004; Wiedermann and von Eye, 2013). Bandura (2006) defined self-efficacy as an individual’s confidence in their ability to perform in a specific area. Further, Bandura (1977) argued that self-efficacy is most often developed through performance events that result in mastery, but may also be developed through observation, persuasion, and events that tax both mind and emotion. While all ECO programs endeavor to build research self-efficacy through such vicarious and performance events, the REU and Summer Institute collect pre- and post-program data to track research self-efficacy. Thus, after the survey data were collected from participants, questions that focused on research objectives (i.e., research reading, writing, presenting, preparation, and mentoring) were analyzed to identify the self-efficacy of participants after completing the NSF NHERI REU or Summer Institute programs [Table 4 (REU) and Table 5 (Summer Institute)].

Table 4

Table 4. NSF NHERI REU research self-efficacy questions and results.

Table 5

Table 5. NSF NHERI summer institute research self-efficacy questions and results.

In terms of the time of programming and intervention, each program is unique in its scope, aim, timeline, and funding, and thus, constitutes its own case study analysis. Between 2017 and 2023, selected REU students, funded by NSF, conducted research for 10 weeks during one summer of their undergraduate careers at one of the NHERI research and experimental facilities. Registered GSC members may elect to participate in monthly meetings, a mini-conference, and various workshops targeting graduate student members throughout the academic year (i.e., August-June). Five NHERI GSC members were selected each year to attend the Summer Institute as NSF-funded participants in 2022-2023. Also, between 2017 and 2023, NSF-funded Summer Institute early-career faculty participated in a three-day intensive workshop. The experience of each group of participants in the educational outreach efforts differed and was targeted to impact members’ performance and vicarious experience positively, incorporating them into the natural hazards engineering research community.

4 Program impact: broadening participation

One main objective of the NHERI ECO programs is to broaden participation of URGs in STEM with a specific focus on civil engineering and interdisciplinary hazards research. The NHERI ECO uses strategic recruitment, holistic application review, diversity, equity, and inclusion awareness and programming, and multiple program components aimed at supporting a diverse cohort of participants, i.e., skill building, mentoring, socialization, supplemental learning, bridge programs, and funding (Palid et al., 2023), to reach this program objective. These programs also included methods to improve research self-efficacy through performance events, observation, and cognitive and emotional engagement (Bandura, 1977). These methods were intentionally selected to support participants since increased self-efficacy is known to predict STEM intentions which are also linked to persistence in STEM as well as supporting URG engagement (Estrada, et al., 2011).

The REU, GSC, and Summer Institute strategic recruitment methods include emailing individuals in departments and professors in ABET-accredited, Minority Serving Institutions (MSIs) and universities outside of Carnegie R1 institutions to reach underrepresented and underserved STEM populations and encourage application submissions and participation. This list includes emails for professors within civil engineering departments of MSIs and all NHERI components. In 2021, a social science list was also created to recruit participants for participation at the NSF NHERI CONVERGE Social Science Research Facility. CONVERGE supported REU participants in 2021 and 2023. This list was created with the help of CONVERGE representatives and included social scientists who focused on natural hazards and disaster resilience research. Finally, in 2024 the GSC began actively recruiting members from a small pilot group of international universities whose names were gathered from NCO and GSC leadership. Once the efforts are analyzed, the GSC may expand the international recruitment campaign.

The NHERI ECO and ECO Committee also trained the selection committee on the holistic review process. By 2023, this included consideration of seven diversity measures, along with other more traditional criteria, e.g., GPA, for recruitment and selection of ECO program participants. The seven demographic considerations included 1) race/ethnicity; 2) gender; 3) first-generation college student status; 4) geographic location; 5) home university Carnegie Classification; 6) previous research experience; and 7) native language. Initial considerations included the first four demographic items but were expanded to include all seven by the fall of 2022.

Beginning in the fall of 2022, NHERI ECO programming also consisted of diversity, equity, and inclusion workshops within the REU, GSC, Summer Institute, ECO Committee Meetings, NHERI-sponsored research summits, and multiple program components. The diversity, equity, and inclusion programming demonstrated NHERI’s commitment to diverse cohorts and inclusive practices. These programs strengthened diverse representation across all flagship events and supported an understanding of the holistic review of applications and participation across the network. After conducting a literature review of 82 STEM intervention programs, Palid et al. (2023) argued that programs that used multiple components to support URGs “likely increase[d] participant success.” Since persistence is often seen as a challenge within STEM fields, the NHERI ECO defines success as persistence in a STEM field or program whether in education or occupation. To support this definition of success, NHERI ECO programs offered all six program components identified in the review (i.e., mentoring, skill building, supplemental learning, socializing, bridge programs, and funding) across its programs to better support all participants. These components also support self-efficacy through the intentional use of performance events, observation/modeling opportunities, and cognitive and emotional engagement (Bandura, 1977).

4.1 REU summer program demographic data

Since 2017, NHERI ECO has hosted six REU cohorts, totaling 166 undergraduate research participants. Seventy-one percent of participants self-identified as civil engineering (i.e., civil engineering, civil and environmental engineering, and structural engineering) majors and other participants identified as 7% mechanical engineering, 5% architectural engineering, 4% environmental engineering, 3% computer science, 2% geology, 2% mathematics, 1% aerospace engineering, 1% anthropology, 1% biological systems engineering, 1% computer engineering, 1% electrical engineering, 1% engineering, 1% environmental analysis, 1% geotechnical engineering, 1% sociology, 1% urban planning. Each January, all REU cohort data is updated to provide a longitudinal view of participants’ progress which begins a semester after completing the REU Summer Program. This paper provides a snapshot of the ongoing longitudinal data from participants from 2017–2023. Of the program alumni, 64% earned their undergraduate degrees, 34% remained in their undergraduate program, and 2% did not respond or found on LinkedIn. These participants were classified as having an unknown status. All accessible participants tracked longitudinally via LinkedIn, have completed or remained in STEM degree programs as of January 2024. Over a third of the REU alumni (35%) have pursued graduate degrees, and 8% were pursuing a doctoral degree in STEM at the time of publication.

Table 1 shares the demographic breakdown of the 166 undergraduate students who participated in the REU Summer Program. The impact of the REU program in broadening participation for URGs in civil engineering from 2017–2023 (except 2020 when no program was offered due to the COVID-19 pandemic) is reflected in its participant make-up, success, and self-efficacy.

The average percentage of students who identified as members of an underrepresented group in STEM, specifically civil engineering, from the REU program include 13.4% Black, 19.5% Hispanic, and 0.6% American Indian or Alaskan Native (AIAN). According to the American Society for Engineering Education (2022) national statistics of students enrolled in engineering programs, only 5.4% identify as Black, 15.8% as Hispanic, and 0.3% as AIAN; therefore, the REU program is increasing the participation percentages compared to the national average. The program aims to expand the representation across typical populations to provide a positive research experience for all participants. Similarly, while 57.3% of REU participants self-identified as women, only 24.2% of engineering undergraduate degrees and 35.6% of civil engineering undergraduate degrees were awarded to women nationally (American Society for Engineering Education, 2022).

Research has shown that diverse populations can provide better solutions to the problems society faces (Hong and Page, 2004), so it is also important to consider other areas of diversity when working to broaden participation. This includes first-generation students currently enrolled in a bachelor’s program whose parents or guardians did not earn a four-year degree (RTI International, 2024); students who attend universities not categorized as Carnegie R1 institutions which may be less likely to offer undergraduate research opportunities; and geographic location.

While NHERI ECO tracks a number of diversity measures, it is more difficult to place these three categories into a larger context. For example, the Center for First-Generation Student Success 2018 report defines a first-generation student as an “undergraduate student whose parents do not have a bachelor’s degree” (RTI International, 2024). Although this is the first-generation definition utilized by the NHERI ECO, the definition of first-generation varies by institution. Only 73% of universities define first-generation college students (Whitley, et al., 2018). According to the First-Gen Forward fact sheets produced by RTI International (2023), 54% of all undergraduate students in 2019–2020 identified as first-generation college students. But when looking specifically at engineering students approximately 3.9% of first-generation college students majored in engineering between 1992 and 2000 (Chen and Carroll, 2005). Of the 166 REU participants from 2017–2023, 27% identified as first-generation college students.

Geographic location is another criterion utilized in participant selection as it addresses underserved populations or areas. In 2017, the Established Program to Stimulate Competitive Research (EPSCoR) was created to fund research in 27 jurisdictions (i.e., states, territories, and commonwealths) that received less federal research funding (U.S. National Science Foundation, 2024). These specific geographic underserved locations receive targeted funding to encourage STEM workforce development and research. EPSCoR is also part of the NSF Broadening Participation Portfolio designed to expand STEM opportunities (NSF, n.d.). REU students from 2017–2023 originated from 37 U.S. states, territories, and the District of Columbia. Of the 166 REU participants, 23% attended school in an EPSCoR jurisdiction. By supporting students from EPSCoR, NHERI ECO continues NSF’s work in building more pathways for students to pursue STEM careers.

Finally, as an NSF-funded program, ensuring funding is equitable across states and universities and the students supported is critical. This equitable distribution also helps to ensure diversity in thought. In 2023 the NHERI ECO began tracking and considering Carnegie Classification as part of the holistic selection process of REU applications. NSF encourages REU programs to support students from institutions that may not receive the same research opportunities. From 2017–2023, 39% of REU participants attended a university not categorized as Carnegie R1 (very high research activity as defined by the Carnegie classification system) (ACE, n.d.). Carnegie R1 research universities support robust research experiences for undergraduates that receive more funding (Jayabalan, et al., 2021). By including the Carnegie Classification as one of the seven categories included in the application review, NHERI hopes to further broaden STEM opportunities for students with fewer undergraduate research opportunities.

4.2 Graduate student council demographic data

The NHERI GSC was launched in October 2021 as a student-led organization to provide virtual programming and mentorship for a community of like-minded graduate students interested in natural hazards research. In the first 2 years of operation (1 October 2021 – 31 July 2023), the GSC registered 385 new members, and the demographic breakdown can be viewed in Table 2. Over half (58%) of the membership originated from countries other than the United States with 41 countries represented. Of the members who are not U.S. citizens, 11% percent are international students studying outside the United States. Doctoral students and candidates made up the largest sector of membership (77%), 19% were master’s students, and 4% were listed as other—comprised of assistant professors, post-doctoral scholars, and undergraduate students. Overall, the NCO-ECO team was pleased with this distribution of membership as it demonstrates pathways for master’s students interested in pursuing research-related degrees and significant support for doctoral students seeking entrance into research positions.

Of the GSC members in the first two years, 12% identified as Black, and 8% identified as Hispanic. The GSC membership compares favorably to the national statistics of engineering doctoral degrees awarded which reported 3.9% Black and 7.5% Hispanic (American Society for Engineering Education, 2022). In addition, when NHERI GSC membership is compared to the national statistics of U.S. science and engineering faculty with 2.5% Black and 3.9% Hispanic (American Society for Engineering Education, 2022), the members show promise for broadening representation for future faculty positions and the next-generation STEM workforce. The graduate student organization reported that 39% of members identified as female compared to the 26.2% of females who were awarded engineering doctoral degrees and the 18.6% of females who were employed as science and engineering faculty (American Society for Engineering Education, 2022).

Thirty-five percent (35%) of GSC members identified as the first in their family to earn an undergraduate degree. However, many graduate schools or surveys use a different definition of first-generation college students. As mentioned earlier, all NHERI ECO programs define a first-generation individual as a student whose parents or guardians did not complete a four-year degree. One brief, Understanding Career Pathways for Program Improvement, using data from the Council of Graduate Schools, defined first-generation as a student who is the “first in their generation to earn a bachelor’s degree and pursue a doctoral degree” (Mitic, 2022). The brief reported that 23% of engineering doctoral students identified as first-generation college students, which again demonstrates the diversity of the GSC.

Since the NHERI GSC accepts members from institutions around the world, its focus is on the number of countries represented rather than states. Members hail from 41 countries, and over half of the members reported citizenship outside the U.S. (58%). American Society for Engineering Education (2022) reported that 57% of the engineering doctoral program enrollment comes from countries other than the United States. Of the international members, 10% are enrolled in institutions outside the U.S. This includes members of the GSC leadership team who attend institutions in Hungary, Nepal, Nigeria, and Iran.

4.3 Summer institute demographic data

NHERI’s Summer Institute has hosted six cohorts of participants, totaling 129 early career faculty and researchers from 40 U.S. states, territories, and the District of Columbia since 2017. The Summer Institute participants represented 13 EPSCoR jurisdictions with 22% of participants enrolled or employed at an institution within an EPSCoR jurisdiction. Pivoting from the in-person Summer Institute, a virtual-only program was held in 2020 due to the COVID-19 pandemic and was open to all members of the community for participation; because of this, no demographic data were available for 2020 Summer Institute participants. Demographic data from participants is identified in Table 3. Early career participants included senior-level graduate students, postdoctoral fellows, researchers, and assistant professors; individuals are tracked longitudinally through LinkedIn, Google Scholar, and NSF Award Search.

Of the Summer Institute alumni, 6.8% identified as Black, and 10.6% identified as Hispanic, compared to 2.5% Black and 3.9% Hispanic of U.S. science and engineering faculty (American Society for Engineering Education, 2022) or the 5% Black and 9% Hispanic for individuals above the age of 25 in engineering positions, according to the Pew Research Center (Fry, et al., 2021). Further, 49% of Summer Institute participants identified as women compared to 18.6% of science and engineering faculty (American Society for Engineering Education, 2022) or 15% of engineers and architects who identified as women (Fry, et al., 2021) Thirty-six percent (36%) of Summer Institute alumni also identified as the first in their family to complete a four-year degree.

Finally, of the Summer Institute participants, 35% of postdoctoral scholars and graduate students secured tenure-track positions, compared to the 12.8% of engineering doctoral graduates who Larson et al. (2013) calculated from 2011 ASEE data can secure a tenure-track faculty position based on the number of graduates and faculty members and the current rate of growth of engineering departments. Similarly, Roy, et al. (2024) expanded Larson et al. (2013) work to include ASEE data from 2006–2021 and found 12.4% of doctoral engineering graduates are likely to secure a tenure-track faculty position. At the time of publication, Summer Institute alumni amassed $39.7 million in NSF funding, published 2,392 articles, won seven prestigious NSF Faculty Early CAREER Awards, and obtained one NSF Graduate Researcher Fellowship Program Award after participating in the Summer Institute.

5 Program impacts

Besides bringing together a diverse group of participants, the NHERI REU and Summer Institute programs provided educational opportunities for them to expand their career experiences. Because the programs were intentionally designed to provide targeted learning outcomes for each audience, it was expected that shifts in learning metrics and research self-efficacy would be achieved. Below is a report of some of the highlights of the learning outcomes and educational impacts for six cohorts of the REU Summer Program and the Summer Institute from 2017–2023.

5.1 REU educational impacts on research self-efficacy

While research experiences supported by specific program components are shown to support members of URGs, self-efficacy has also been found to have a positive impact on URG’s STEM outcomes (Palid, et al., 2023). Thus, the final NHERI REU table shares the pre- to post-assessed research self-efficacy of the 2017–2023 NHERI REU participants which were analyzed using a paired samples t-test in SPSS 27 (Table 4). The table compares the mean (M) and standard deviation (SD) of change in research self-efficacy. The longitudinal undergraduate participant data shows a statistically significant growth in research self-efficacy in each key area after a 10-week REU experience.

REU participants showed a statistically significant increase in self-efficacy in understanding theory and concepts (M = 1.10, SD = 1.27), t(154) = 10.82, p < 0.0005 d = 0.87, journal articles (M = 1.05, SD = 1.24) t(154) = 10.54, p < 0.0005, d = 0.85, professional data and research presentations (M = 1.13, SD = 1.18) t(154) = 11.89, p < 0.0005, d = 0.96, relevance of research to coursework (M = 0.88, SD = 1.35), t(154) = 8.16, p < 0.0005, d = 0.66, and what is involved in everyday research (M = 1.70, SD = 1.23), t(154) = 17.18, p < 0.0005, d = 1.38. Participants also demonstrated a statistically significant improvement in confidence in engaging in hands-on research (M = 1.62, SD = 1.29), t(154) = 15.62, p < 0.0005, d = 1.26, writing papers (M = 1.32, SD = 1.22), t(154) = 13.48, p < 0.0005, d = 1.08, creating and preparing posters (M = 1.68, SD = 1.44), t(154) = 14.60, p < 0.0005, d = 1.17, and discussing scientific concepts with those outside the field (M = 1.41, SD = 1.38), t(154) = 12.73, p < 0.0005, d = 1.02. Finally, the data also shows that participants experienced a statistically significant amount of quality mentorship (M = 1.02, SD = 1.61), t(154) = 7.91, p < 0.0005, d = 0.64 and mentor collaboration (M = 1.57, SD = 1.61), t(154) = 12.08, p < 0.0005, d = 0.97 during their research experience. The effect size of all results ranged from medium (0.50–0.79) to large effect size (0.80+) as shown in Cohen’s d column in Table 4 (Cohen, 1988).

5.2 Summerns institute impacts on knowledge and research self-efficacy

Summer Institute participants’ knowledge and self-efficacy show statistically significant improvement for all self-efficacy questions except Q4 and Q15. Pre- and post-assessment data demonstrate that participants developed additional experience working with the natural hazards engineering community in general (M = 1.78, SD = 1.01), t(123) = 20, p < 0.0005, d = 1.26, within their specialization (M = 0.71, SD = 1.34), t(123) = 6, p < 0.0005, d = 0.87, and outside their specialization (M = 1.50, SD = 1.27), t(123) = 13, p < 0.0005, d = 0.66. Although there was a mean increase between pre- and post-assessment, there was not a statistically significant change in participants’ interest in collaborating with other natural hazards engineering professionals (M = 0.40, SD = 0.93), t(123) = 0.05, p = 0.631 d = 1.38, nor learning about the NHERI network, the research work, and resources (M = -0.09 SD = 0.88), t(123) = −1, p = 0.266, d = 0.96. Participants reported a statistically significant increase in knowledge about the NHERI network and its resources (M = 1.40, SD = 1.05) t(123) = 15, p < 0.0005, d = 0.85, feelings of preparedness to write a proposal supplementing or extending their research (M = 0.90, SD = 1.10), t(123) = 9, p < 0.0005, d = 1.08, with researchers outside their university (M = 0.59, SD = 1.18), t(123) = 6, p < 0.0005, d = 1.17, and with researchers outside their specialization (M = 0.51, SD = 1.18), t(123) = 5, p < 0.0005, d = 1.02. All questions’ effect sizes also ranged from medium (0.50–0.79) to large effect sizes (0.80+) as shown in Table 5 under Cohen’s d (Cohen, 1988).

6 Conclusion

The NHERI ECO, ECO Committee, and NHERI research and experimental facilities collaborated to identify pathways to broaden participation for underrepresented populations in STEM and natural hazards engineering research. The NHERI community used targeted recruitment efforts for participants at MSIs, holistic application review, diverse and inclusive training and programming, and multiple supportive program components (Palid et al., 2023) as well as intentional programming that included performance mastery and vicarious learning experience (Bandura, 1977) to engage the whole individual academically, personally, and socially. These focused activities helped to open various, diverse pathways that support undergraduates, graduate students, and early-career faculty and connect them with accomplished NHERI researchers and scholars. These pathways help broaden participation in STEM and natural hazards engineering research while expanding the interdisciplinary knowledge needed to effectively mitigate future natural hazards. The demographic, longitudinal, and pre-assessment and post-assessment data demonstrate the varied pathways that participants take within the natural hazard workforce. These data also provide the impact of knowledge gained through participation in established programs. The NHERI ECO, ECO Committee, and the NHERI research and experimental facilities worked to deliberately create a community supporting the next-generation of natural hazards researchers.

For engineering educators interested in preparing the next-generation of diverse workforce and faculty (American Society of Engineering Education and National Academy of Engineering, 2024), it is critical to enlist the expertise of equity-focused education and engineering education researchers. This can be done by collaborating with engineering education experts on program design, recruitment, holistic selection, implementation, and assessment, or by training to learn the many ways engineering education research can improve programs. During the formative stages of the REU and Summer Institute programs, lesson plans were created in partnership with education experts that included learning objectives, activities designed to engage participants, and assessments connected to the grant’s goals and the established learning objectives. In this way, the focus of the education outreach activities was meaningful and targeted toward equity by design. These actions remain essential to creating pathways where people from all backgrounds feel welcomed and included. Their ideas and ways of problem-solving can lead to more innovations in the much-needed and evolving natural hazards engineering fields.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Shannon Marquess and Tammy Lopez, University of Texas at San Antonio. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

RN: Formal Analysis, Investigation, Writing–original draft, Writing–review and editing. KV: Conceptualization, Supervision, Writing–original draft, Writing–review and editing. JB: Project administration, Supervision, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. The programs and research are based upon work supported by the U.S. National Science Foundation under the Natural Hazards Engineering Research Infrastructure Network Coordination Office Award No. 2129782.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declare that no Generative AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

American Council on Education (ACE) (2024). “Carnegie classification of institutions of higher education,” in Carnegie classification. Available at: https://carnegieclassifications.acenet.edu (Accessed September 25, 2024).

Google Scholar

American Society for Engineering Education (ASEE) (2022). Profiles of engineering and engineering technology. Available at: https://ira.asee.org/wp-content/uploads/2024/03/Engineering-and-Engineering-Technology-by-the-Numbers-cover-combined.pdf (Accessed May 10, 2024).

Google Scholar

American Society of Engineering Education and National Academy of Engineering (ASEE and NAE) (2024). The inclusive engineering mindset. Available at: https://mindset.asee.org/wp-content/uploads/2024/09/The-Engineering-Mindset-Report.pdf (Accessed September 30, 2024).

Google Scholar

Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev. 84 (2), 191–215. doi:10.1037/0033-295X.84.2.191

PubMed Abstract | CrossRef Full Text | Google Scholar

Bandura, A. (2006). “Guide for constructing self-efficacy scales,” in Self-efficacy beliefs of adolescents. Editors F. Pajares, and T. C. Urdan (Greenwich, CT: Information Age Publishing), 5, 307–337.1.

Google Scholar

Chen, X., and Carroll, C. D. (2005). “First-generation students in postsecondary education: a look at their college transcripts,” in Nces 2005-171. U.S. Department of Education, National Center for Education Statistics. Washington D.C.: U.S. Government Printing Office.

Google Scholar

Cohen, J. W. (1988). Statistical power analysis for the behavioral sciences. 2nd Edn. Hillsdale, NJ: Lawrence Erlbaum Associates.

Google Scholar

Estrada, M., Woodcock, A., Hernandez, P. R., and Schultz, P. W. (2011). Toward a model of social influence that explains minority student integration into the scientific community. J. Educ. Psychcol. 103 (1), 206–222. doi:10.1037/a0020743

PubMed Abstract | CrossRef Full Text | Google Scholar

Fradette, K., Keselman, H. J., Lix, L., Algina, J., and Wilcox, R. R. (2003). Conventional and robust paired and independent-samples t tests: type I error and power rates. J. Mod. Appl. Stat. Methods 2 (2), 481–496. doi:10.22237/jmasm/1067646120

CrossRef Full Text | Google Scholar

Fry, R., Kennedy, B., and Funk, C. (2021). STEM jobs see uneven progress in increasing gender, racial, and ethnic diversity. Washington D.C.: Pew Research Center. Available at: https://www.pewresearch.org/social-trends/2021/04/01/stem-jobs-see-uneven-progress-in-increasing-gender-racial-and-ethnic-diversity/ (Accessed December 17, 2024).

Google Scholar

Hong, L., and Page, S. E. (2004). Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Econ. Sci. 101 (46), 16385–16389. doi:10.1073/pnas.0403723101

PubMed Abstract | CrossRef Full Text | Google Scholar

Jayabalan, M., Caballero, M. E., Cordero, A. D., White, B. M., Asalone, K. C., Moore, M. M., et al. (2021). Unrealized potential from smaller institutions: four strategies for advancing STEM diversity. Cell 184, 5845–5850. doi:10.1016/j.cell.2021.10.030

PubMed Abstract | CrossRef Full Text | Google Scholar

Larson, R. C., Ghaffarzadegan, N., and Xue, Y. (2013). Too many PhD graduates or too few academic job openings: the basic reproductive number R0 in academia. Syst. Res. Behav. Sci. 31, 745–750. doi:10.1002/sres.2210

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitic, R. R. (2022). Insights into first-generation doctoral Students. PhD career Pathw. A Proj. Counc. Graduate Sch. Available at: https://cgsnet.org/wp-content/uploads/2022/03/CGS_CP_First-Gen-Doc-Students_ForWeb.pdf (Accessed September 29, 2024).

Google Scholar

National Academies of Science, Engineering and Medicine (NASEM) (2016). Barriers and opportunities for 2-Year and 4-Year STEM degrees: systematic change to support students’ diverse pathways. Washington, DC: The National Academies Press. doi:10.17226/21739

CrossRef Full Text | Google Scholar

Palid, O., Cashdollar, S., Deangelo, S., Chu, C., and Bates, M. (2023). Inclusion in practice: a systematic review of diversity-focused STEM programming in the United States. Int. J. STEM Educ. 10, 2. doi:10.1186/s40594-022-00387-3

CrossRef Full Text | Google Scholar

Rasch, D., and Guiard, V. (2004). The robustness of parametric statistical methods. Psychol. Sci. 46 (2), 175–208. doi:10.1073/pnas.0403723101

CrossRef Full Text | Google Scholar

Roy, S., Velasco, B., and Edwards, M. A. (2024). Competition for engineering tenure-track faculty positions in the United States. PNAS Nexus 3 (5), pgae169. doi:10.1093/pnasnexus/pgae169

PubMed Abstract | CrossRef Full Text | Google Scholar

RTI International (2023). First-generation college students in 2020: demographic characteristics and postsecondary enrollment. Washington, DC: FirstGen Forward. Available at: https://firstgen.naspa.org/files/dmfile/15405_NASPA_FactSheet-01.pdf (Accessed December 11, 2024).

Google Scholar

RTI International (2024). First-generation college graduates after four years: experiences during the COVID-19 pandemic: 2020 to 2021. Washington, DC: NASPA. Available at: https://firstgen.naspa.org/files/dmfile/15405_NASPA_FactSheet-041.pdf (Accessed December 11, 2024).

Google Scholar

Singer, J. D., and Willet, J. B. (2003). Applied longitudinal data analysis: modeling change and event occurrence. Oxford University Press. doi:10.1093/acprof:oso/9780195152968.001.0001

CrossRef Full Text | Google Scholar

U.S. National Science Foundation (2024). NSF established program to stimulate competitive research (EPSCoR). Available at: https://new.nsf.gov/funding/initiatives/epscor (Accessed December 11, 2024).

Google Scholar

Whitley, S. E., Benson, G., and Wesaw, A. (2018). First-generation student success: a landscape analysis of programs and services at four-year institutions. Cent. First-generation Student Success, NASPA–Student Aff. Adm. High. Educ. Entangled Solutions. Available at: https://firstgen.naspa.org/files/dmfile/NASPA-First-generation-Student-Success-FULL-REPORT.pdf (Accessed December 11, 2024).

Google Scholar

Wiedermann, W., and von Eye, A. (2013). Robustness and power of the parametric t-test and the nonparametric Wilcoxon test under non-independence of observations. Psychol. Test. Assess. Mod. 55 (1), 39–61.

Google Scholar

Yin, R. K. (2009). Case study research: design and methods. Thousand Oaks, CA: Sage Publications, Inc.

Google Scholar