Elizabeth J. Salm
Caleb C. McKinney- 1Biomedical Graduate Education, Georgetown University Medical Center, Washington, DC, United States
- 2Biomedical Graduate Education and Department of Rehabilitation Medicine, Georgetown University Medical Center, Washington, DC, United States
A growing number of STEM doctorates pursue careers across the broader biomedical workforce, including industry, policy, and healthcare. Graduate and postdoctoral trainees need training to develop professional skills that prepare them for diverse workforce options. Through engagement with faculty and trainees, we determined that formal management skills are underdeveloped in trainees, particularly around managing projects in a way that is translatable to broader industries. At Georgetown University Medical Center, we adapted Kern's six-steps of curriculum design to develop the Academy for Transferable Management Skills (ATMS) program to help graduate and postdoctoral trainees develop linear experience in utilizing project management tools in their academic research contexts. ATMS includes a self-paced online CANVAS course with learning objectives and content modules that map to the project management cycle from initiation to closure, developed in consultation with PhD-level industry experts. From 2021–2024, 25 trainees have completed the ATMS program, including the capstone project and posttest evaluation. Trainees also complete brief quizzes after each module as a formative assessment of learning. The pre-test evaluation (n = 92) revealed a baseline of project management “pain points” regularly encountered by trainees (risk management, project charters, work breakdown structures, and managing project scope). Posttest data (n = 25) reveal a significant increase (p < 0.0001) in project management self-efficacy measures across the aforementioned pain point scales. Notably, 100% of trainees indicated that they may/would refer the program to colleagues. ATMS offers trainees the flexibility to pick frameworks that apply to their projects with trainees planning to use project schedules (84%), Work Break Down Structures (80%), lessons learned reports (68%), and communication plans (68%) in their work. This integrated experiential learning approach equips trainees to develop and execute their projects according to industry-informed project management principles, which allows them to perform their current research more efficiently and to utilize project management frameworks in a way that is directly transferable to broad careers.
1 Introduction
The career landscape for PhD graduates has evolved with an increasing number choosing to pursue non-faculty careers (Doctoral Recipients from U.S. Universities: 2022, 2023). STEM PhDs have sought growing opportunities in pharmaceutical research, teaching, research administration, science policy, consulting, and science communication (Mathur et al., 2018; Xu et al., 2018; Brown et al., 2023). This changing landscape of PhD career options suggests that our current apprenticeship model more focused on developing future faculty needs renovation so that training in research design, publications, and technical skills is augmented with the development of broader workforce-ready skills that are useful in many professional contexts.
During PhD and postdoctoral training, trainees develop a variety of skills throughout courses and their research training that are broadly transferable to many careers (Sinche et al., 2017; Ganapati and Ritchie, 2021). Beyond discipline specific-knowledge, STEM trainees hone their written communication through dissertations, reports, and publications. Opportunities to present at conferences and in departmental seminars develop their oral communications skills. While navigating experimental results, they enhance their problem-solving skills as well as data analysis and interpretation abilities.
However, employed PhD alumni report gaps in skills needed for their work that were not developed during their PhD training (Sinche et al., 2017; Ganapati and Ritchie, 2021). Alumni have indicated that their training did not adequately prepare them for managing others, working on a team or with external collaborators, or managing their time effectively. Those skills are essential components of project management and are especially important across biomedical job sectors where team-based projects are prevalent (Mason et al., 2016; Muurlink et al., 2024). Beyond teamwork and management, alumni also commonly mention lack of career planning and awareness of career options as a weakness in PhD training programs.
In the wake of the NIH Broadening Experiences in Scientific Training programs (Lenzi et al., 2020), biomedical PhD career development has expanded across institutions to increase opportunities for career exploration. Particularly, direct employer engagement through internships, job shadowing, and employer site visits provide experiential learning opportunities that introduce trainees to a variety of careers (Chatterjee et al., 2019; Van Wart et al., 2020; Collins et al., 2022; Brandt et al., 2023). Moreover, faculty mentors acknowledge the importance of institutional career development activities in augmenting their roles in supporting the career advancement of their trainees, but they often feel that they do not have the requisite expertise needed to mentor exploration and skill development across career options (Watts et al., 2019).
Integrating the perspectives of internal stakeholders (trainees, faculty, and administrators) and external industry professionals is pivotal toward maximizing the ability of career and professional development initiatives to link the experiences of trainees to the skills needed to pursue broad career options (Ramadoss et al., 2022). At Georgetown University Medical Center, we consulted with students, faculty, and external industry partners to design the Academy for Transferable Management Skills (ATMS) program to bring workforce-ready skills “from the outside in,” allowing graduate students and postdoctoral fellows to leverage their current research contexts to learn and showcase project management competencies that are widely used across careers (Project Management Institute, 2017).
2 Pedagogical framework and learning environment
We adapted Kern's six-steps of curriculum design (Thomas et al., 2022) to develop the Academy for Transferable Management Skills (ATMS) program to help graduate and postdoctoral trainees develop linear experience in utilizing project management tools in their academic research contexts:
2.1 Steps 1 and 2: problem identification and needs assessment
A careful review of the literature as presented above has revealed that the training culture needs to shift to match the changing landscape of careers, and that there is potential for integrating creative educational interventions that teach cross-functional workforce skills within the academic context. We conducted one-on-one interviews with a small sampling of faculty members across the departments of microbiology, oncology, and neuroscience. We also conducted a focus group of PhD students and postdoctoral trainees across the departments of oncology, neuroscience, pharmacology, and pediatrics. The faculty interviews and the trainee focus group revealed specific pain points to target to facilitate career exploration and development of workforce skills for trainees, and provided ideas for implementation. Faculty and trainees noted the importance of project management skills within academia and the broader scientific workforce, and mentioned the dearth in opportunities for project management training in academic research. Overall, trainee challenges centered around staying organized, planning, balancing career development with research, and translating skills gained during academic research to broader workforce settings. Faculty favored an intervention with minimal time commitment, yet thorough enough to provide frameworks for trainees to adapt to their research projects in a way that faculty can understand and follow. Both trainees and faculty were in favor of an online format given the flexibility in delivery.
2.2 Step 3: outlining goals and objectives
Given the themes we identified above, we defined our goal as designing an integrated experiential learning program to teach trainees how to develop workforce ready project management skills that they can not only utilize during their actual training to improve their research productivity, but also how to leverage their research context to showcase direct application of these professional competencies across workplace contexts. This paradigm would also allow trainees to augment external (to the research group) experiential learning opportunities by exploring relevant workforce skills within their current research group.
2.3 Step 4: building educational strategies
We leveraged the design thinking process (Waidelich et al., 2018) to build our educational strategies and design a prototype for testing before implementing more broadly across our trainee community. During an ideation workshop with industry professionals, we engaged in a series of brainstorming prompts to curate possible workforce competencies to train based on feedback from our needs assessment stage. Ultimately, our concepts mapped onto phases of the project management cycle, which formed the outline for the ATMS course. We built a prototype course in the CANVAS learning management system based on the content generated in the ideation phase that a group of PhD students and Postdoctoral trainees (n = 4) could test. The prototype we built was composed of five modules spanning key phases of project management: initiation, planning, execution and monitoring, risk management, and closing. Each module delivered content in multiple ways to engage the learning experience, including recorded videos, articles, and self-guided slides. After the small group of trainees completed the training, the ATMS team convened them in a focus group meeting to review modifications that could be made to the course to enhance platform usability and further tailor content for application to the trainee's research settings. The trainees indicated that the risk management and project closing modules were too corporate-focused and that the structure and number of assignments in the project planning module was too taxing. We addressed those comments by including additional research-relevant project examples and by reducing the number of assignments in the project planning module.
2.4 Step 5: implementation
After we refined our prototyped ATMS course, we invited doctoral students and postdoctoral researchers across Georgetown University to participate in the finalized ATMS online course. The program is advertised broadly across Georgetown as an optional, non-credit bearing activity, and participants who completed the course and assignments receive a certificate of completion. Of note, in recent years, the course has been adopted as either a requirement or recommended activity in several federally-funded specialized Georgetown University training programs.
2.5 Step 6: evaluation and feedback
We evaluated the effectiveness of the ATMS training according to Reaction (level 1), Learning (level 2), and Behavior (level 3) levels of the Kirkpatrick model for training program evaluation (Kirkpatrick and Kirkpatrick, 2006). At the end of each ATMS module, there is a multiple-choice quiz that assesses knowledge acquisition of the concepts presented in the module that pertain to the relevant phase of project management presented in the content (level 2). Consistent with fostering the collaborative relationship between learner and educator derived from self-directed adult learning frameworks (Garrison, 1997), we designed reflective assignments and capstone portfolio pieces to facilitate and assess the application of project management skills to practice in research training environments and career exploration (level 3). Pre and post surveys were administered online. The pre-training evaluation survey (pre-test) was distributed when trainees registered for the course to establish a baseline of project management skills as well as collect demographic information. The post-training evaluation (post-test) was provided to trainees after completing the program to determing their intent to refer the program to others (level 1), how long it took them to complete the program, and which project management frameworks they planned to bring into their research and career (level 3). Importantly, the post-test also assessed changes in reported self-efficacy in project management skills, an important predictor of project performance (Blomquist et al., 2016). Preliminary results from Step 6 are presented in the Section 4.
3 Learning objectives and pedagogical format
The ATMS curriculum was developed in close consultation with STEM PhD holders who entered project management roles across workforce sectors, including government, consulting, and regulatory affairs. The perspective from their broad professional experience and academic training allowed them to identify important project management concepts and relate them to academic research (Step 4, Pedagogical Framework and Learning Environment). The ATMS model has two main components: (1) asynchronous online modules designed in the CANVAS learning management platform for trainees to learn project management frameworks on their own schedule; and (2) a supplemental menu of activities allowing trainees to develop their project management skills in their own research as well as build a community of practice and engage with other trainees. We built the course online in an asynchronous format to account for the unpredictable nature of research schedules, especially across multiple programs and fields. This online format allows trainees to work at their own pace to easily return to the course materials, a paradigm of flexibility that was particularly important when the COVID-19 pandemic mobilized all curricular activities to virtual formats (Bilal et al., 2022). Furthermore, trainees can choose when they want to work on the course, so the interruption to their own research schedule is minimal. Our current framework allows trainees to join the ATMS course at any point and attend activities throughout their training.
3.1 Overview of the ATMS online module content
The ATMS online course contains five modules that contextualize industry-ubiquitous frameworks and tools across the project management lifecycle (Project Management Institute, 2017) in a research context. The first two modules of ATMS review the first two phases of project management: project initiation and project planning. Module one gives them the template for a project charter as well as the uses for a charter in academia. In module two, trainees learn many project management techniques useful for planning a project's deliverables and tasks. In the third module, trainees learn about project risk management throughout a project lifecycle. The risk management module shows trainees what different risks look like in research as well as how to plan for and track these risks in a risk register. Project execution and project monitoring are discussed in the fourth module, which details how trainees can manage their projects, being mindful that expanding project scope or scope creep can be a huge problem in research projects. The final ATMS module discusses multiple aspects of closing out a project including how to record the lessons you learned from the project. Throughout the modules, trainees learn the official project management definitions, and how they can apply them to their own research projects, expanding their vernacular of industry-validated frameworks using their current research training context. Specific learning objectives for the course are outlined below:
Module 1—Project initiation
• Identify factors that influence your decision to pursue and/or create a specific project.
• Define the purpose and elements of a project charter.
• Identify project stakeholders and their roles with regards to your research project.
• Develop a project charter for your research project.
• Describe criteria to consider when choosing a team.
Module 2—Project planning
• Explain the basic steps of project planning.
• Describe the scope, requirements, and milestones of your project.
• Develop a Work Breakdown Structure (WBS) for your research project.
• Identify the tasks of a project and set task specific objectives, goals, and timelines.
• Implement a communication plan for your project.
Module 3—Risk and opportunity management
• Define risk and opportunity for a scientific research project.
• Identify risks and opportunities that pertain to your current research projects.
• Implement a successful risk management plan using the 6 steps.
• Distinguish a risk contingency vs. risk mitigation plan.
• Create a risk register to track your identified risks.
Module 4—Project execution and monitoring.
• Maximize team morale through team building and communication during your project.
• Provide respectful feedback for members of your project.
• Plan, assess, and make adjustments to ensure the quality of your project.
• Maintain wellness and resilience during a challenging phase.
• Manage changes that occur during your project.
• Identify and prevent scope creep from getting your project off track.
Module 5—Project closing
• Define the purpose and processes of the Project Closing Phase.
• Create a Post-Project Review Checklist.
• Identify lessons you learned from your project.
• Conduct a Post-Project Meeting and prepare a report for stakeholders.
3.2 Participant engagement: learning and application
To maximize engagement with the content, material in ATMS is presented in multiple ways that leverage perspectives within and outside of academia, through written lectures, embedded PowerPoint slides, video lectures, and articles from a variety of perspectives. We also recorded short audio introductions at the beginning of each module that situates that module's material in the context of the trainee's research.
ATMS incorporates 20 short assignments for trainees to apply the knowledge they've learned from the course. Our first type of active learning assignments are reflective exercises in which we ask trainees to look back at experiences in their past that relate to project management. Example thought exercises include identifying stakeholders, reflecting on feedback they've received or how changes to projects were handled in the past, and identifying different contingency and mitigation plans they've used to combat a project risk. There's an additional thought exercise at the end of every module asking trainees to reflect on how they might use the module material in their own research projects. Our goal was to allow the trainees to tailor what they wanted to use from the lesson as implementing all aspects of ATMS could be overwhelming.
Trainees also complete a project portfolio composed of a series of project management frameworks adapted to their own research. These portfolio pieces ask trainees to develop project management documents for a project they are working on. The majority of these pieces are focused in the project initiation and planning modules where there are concrete documents to produce. Examples of portfolio pieces include a project scope statement, Work Breakdown Structure, risk register, and communication plan for their project. The goal of these assignments is to allow trainees to experiment with different frameworks within project management, testing out which can be most useful for their own work. These pieces also push trainees to think concretely about their project and provide documents they can discuss with their PI. Trainees submit their portfolio pieces to receive feedback on how they might improve their use of project management.
To help orient trainees to how these assignments can be completed for a research-related project, we include an example project of applying to a National Institutes of Health (NIH) National Research Service Award (NRSA) individual fellowship throughout the ATMS modules. In each module, we discuss how they can manage an NRSA application process using project management techniques. Trainees see how to use project planning in this context as well as see how to actually apply risk management to your research. By giving the trainees templates and showing them the example project portfolio pieces, we encourage them to put some of these frameworks in place in a project of their choosing.
An important aspect to facilitate learning is the opportunity for feedback on work (Lee and Chiu, 2022). Feedback is given to trainees through multiple mechanisms. For asynchronous feedback, trainees take quizzes at the end of each module, testing their knowledge and comprehension of the material introduced. Trainees can receive feedback on their own use of project management from the ATMS team by submitting their portfolio pieces for review. Informal project management discussions also allow trainees to learn from each other as they discuss concepts like time management, project management online tools, etc.
4 Results to date
4.1 Methods
The Georgetown University Institutional Review Board has granted an exemption to this study: STUDY00002715. Responses to pre and post surveys assessed trainee self-efficacy through self-reported confidence in project management competencies before (pre survey) and after training (post survey) using 24 items on a 7-point rating scale from 1-(strongly disagree) to 7-(strongly agree), with higher scores reflecting higher confidence. Responses were anonymized by the trainee's creation of a unique identifier code in the pre-training evaluation. Survey data were reviewed for the frequencies and averages of responses. A two-tailed student's t-test was used to compare pre-test differences in project management confidence between demographic groups and training levels, and a paired t-test was used to compare changes in project management confidence after training (pre- vs. post-training evaluation data).
4.2 Trainee demographics
Demographic information was asked in the pre-training evaluation, including gender, race/ethnicity, age, and stage of training (Table 1, N = 92). Seventy-two percent of registrants were graduate students, 71% identified as women, and 28% were from race/ethnic groups underrepresented in biomedical research (NOT-OD-20-031: Notice of NIH's Interest in Diversity, 2019).
Table 1. Comparison of baseline self-reported self-efficacy by demographics and training stages of all registered trainees.
4.3 Pre-training evaluation of skills
To establish a baseline of self-efficacy in project management skills, trainees completed a pre-training evaluation survey of 24 questions to assess their confidence level with a variety of project management skills, tools, and frameworks used across the PM lifecycle (N = 92, Figure 1). While trainees were overall somewhat confident in their ability to perform most project management frameworks, we identified six pain points where more than 50% of trainees rated their comfort in their abilities 1-(strongly disagree) to 3-(somewhat disagree) on a seven point Likert scale, and we considered this range to be “not confident.” From the project initiation competencies, 57% of trainees were not confident in their ability to develop a project charter. From project planning, trainees were overall confident in their ability to plan their project, but 54% identified difficulties in developing a work breakdown structure that identifies deliverables and tasks. Trainees were most uncomfortable with project risk management. While they reported confidence in their ability to identify risks, 52% were uncertain how to implement a successful risk management plan. Sixty-four percent were unable to identify the difference between a risk contingency and a risk mitigation plan. Finally, 66% were not comfortable creating a risk register to track their identified risks. Fifty-two percent of trainees also struggled with how to prevent scope creep.
Figure 1. Baseline self efficacy in project management of all registered trainees. Distribution of self-reported self-efficacy ratings for the 24 pre-training evaluation questions on project management skills, tools, and frameworks prior to completing the ATMS course (n = 92). Competencies where more than 50% of trainees rated their confidence as 1-(strongly disagree) to 3-(Somewhat disagree) were identified as pain points: (1) develop project charter, (2) develop work breakdown structure, (3) implement risk management plan, (4) distinguish between risk mitigation and contingency plans, (5) create risk register, and (6) prevent scope creep. Data were visualized using GraphPad Prism 10.2.3.
4.4 Differences in initial self-efficacy between groups
While differences in overall self-efficacy between graduate students and postdocs was not significant, (average overall self-efficacy: students 4.26 ± 0.10 SE; postdocs 4.62 ± 0.18 SE; p = 0.060; Table 1), self-efficacy across the six pain points identified in the general analysis was statistically lower in graduate students compared to postdocs (average pain point self-efficacy: students 3.12 ± 0.13 SE, postdocs 3.92 ± 0.22 SE p = 0.002). Risk management was of particular concern to students. Fifty-eight percent, 68%, and 73% of students lacked confidence in managing risk, differentiating between contingency and mitigation plans, and creating a risk register respectively compared to 33%, 47% and 43% of postdocs. Handling scope creep was another area of concern for students. Students were less confident than postdocs in preventing scope creep (Not confident: students 61%, postdocs 27%).
While differences in career self-efficacy between demographic groups have been previously reported (Chatterjee et al., 2019), we did not see a difference in overall confidence scores between self-identified women and men across the project management competencies assessed (average self-efficacy rating: women 4.37 ± 0.10 SE; men 4.35 ± 0.16 SE; p = 0.908). When comparing comfort in identified pain points between trainees from racial/ethnic groups underrepresented in biomedical research (NOT-OD-20-031: Notice of NIH's Interest in Diversity, 2019) and trainees from well represented groups (WR: White and Asian), there was no significant difference across the project management competencies assessed (average self-efficacy rating: WR 4.42 ± 0.09 SE; UBR 4.26 ± 0.16 SE; p = 0.417), nor in the identified pain points (average pain point self-efficacy rating: WR 3.47 ± 0.12 SE, UBR 3.06 ± 0.20 SE; p = 0.114).
4.5 Post-training evaluation of project management skills
To measure changes in PM self-efficacy from baseline, trainees were asked to complete a post-training evaluation survey after completing the course, self-reporting their confidence in the same 24 PM competencies from the pre-training evaluation. A total of 25 trainees (27% response rate; seven postdocs and 18 PhD students) completed both pre-training and post-training evaluations. Response rates were similar among demographic groups except 35% of trainees from racial/ethnic groups underrepresented in biomedical research completed both evaluations vs. 28% of trainees from well-represented groups (Table 1). Comparing pre-training to post-training evaluations, trainees reported statistically increased confidence in each of the 24 project management competencies (Figure 2 and Table 2). Notably, confidence scores across the six pain points identified in the pre-training evaluation (Figure 3) were increased (p < 0.0001). Excerpted from one trainee's feedback:
“I really appreciate the opportunity to attend this training. It has helped me to better understand the concepts I already knew and build on those and learn new concepts and tools that I was not aware of in term[s] of managing a project.”
Figure 2. Project management self efficacy of trainees after completing ATMS program. Distribution of self-reported self-efficacy ratings for the 24 post-training questions on project management skills, tools, and frameworks after completion of the ATMS course (n = 25). Data were visualized using GraphPad Prism 10.2.3.
Table 2. Statistical results for pre- and post-training evaluations of trainees who completed the ATMS program.
Figure 3. Significant alleviation of project management pain points in trainees who completed the ATMS program. Trainees who completed ATMS reported a significant increase in self-efficacy for the six project management pain points established by pre-training evaluation data (n = 25). Data were analyzed and visualized using GraphPad Prism 10.2.3. ****p < 0.0001.
Among the trainees who completed the course, there was no significant difference in overall confidence scores between graduate students and postdocs after the ATMS training (students 6.27 ± 0.08 SE, postdocs 6.19 ± 0.10 SE; p = 0.618), nor in the isolated pain point competencies (students 6.07 ± 0.11 SE, postdocs 5.95 ± 0.11 SE; p = 0.54).
We also asked trainees to indicate which project management frameworks they wanted to bring into their own research. Eighty-four percent indicated that they planned to use the project scheduling frameworks with 80% planning to incorporate a Work Breakdown Structure (WBS). Sixty-eight percent of trainees plan to use the lessons learned report and the communication plan. More than 50% of trainees want to use a project charter, risk register, and a 7-point project closing checklist in their research.
On average, trainees took 18 h to complete the course, and 100% may/would recommend the training to another trainee. Another trainee noted:
“the combination of the materials, workshops and videos are an extraordinary source of knowledge and easy to understand”
5 Discussion
Our results in this pilot study show the effectiveness of a human-centered approach to career development program design. ATMS trainees reported improved self-efficacy for many project management skills. They also indicated an excitement to use the project management frameworks in their own research. Overall, this pilot study indicates the importance of designing educational programs that consider broad perspectives from stakeholders and maximal user engagement.
5.1 Trainee-centered program design
ATMS was developed with trainee and faculty input and provides an overview of project management frameworks and tools for research trainees. As we expand the ATMS program into discernable cohorts, the evaluation design allows for the fresh identification of pain points for each new ATMS cohort, followed by just-in-time programmatic interventions to target those pain points through curriculum modifications and supplemental ATMS activities. For example, one major area of concern for trainees was their ability to manage risk and other related skills. With that information, we were able to supplement course material on risk management with activities that highlighted important concepts. One weekly activity focused on distinguishing between risk contingency and mitigation planning. The audio introduction to risk management focused on the usefulness of a risk register for a research lab. Finally, we presented a 30-min risk management workshop incorporating a discussion of risk in two scenarios.
Identifying and preventing scope creep was another area of concern for trainees. Issues around staying organized during a project were also identified by faculty during our initial needs assessment and reflect some aspects of managing project scope. Therefore, we carefully designed the course content to address the ramifications of scope creep in academic research through the initial audio introduction, an additional video detailing an example of scope creep, and supplemental workshops and activities that bring trainees together to discuss and ask questions about this topic and other project management pain points.
5.2 Integrated experiential learning and career development
One important feature of ATMS is the integration of career development with trainee's research priorities. We've termed this process integrated experiential learning because trainees adapt industry-validated project management skills to their current research projects, framing the management of their projects in a way that is more directly translatable across broad careers. Consistent with competency-based learning (Frank et al., 2010), trainees can tailor the course to their goals by applying project management competencies directly to their current work and then receiving feedback. Through the optional project management discussions, trainees also have the opportunity to interact and learn from each other, creating a community of practice centered around research project management. Furthermore, ATMS teaches trainees that they can treat anything as a project, including career development and the job search process. Another important implication for career development is the wide applicability of the project management skills taught in ATMS across career fields. Careers for PhD holders often involve a lot of planning, self-management, and collaboration (Sinche et al., 2017). Project management skills are also beneficial when starting as a new principal investigator (Burroughs Wellcome Fund Howard Hughes Medical Institute, 2006), and ATMS introduces trainees to frameworks that can be incorporated into management structures for research groups.
5.3 Areas for program expansion and future study
While our current phase of evaluation assesses the reaction, learning, and behavior levels of Kirkpatrick's model for program evaluation (Kirkpatrick and Kirkpatrick, 2006), we plan to assess the more longitudinal results level through long term surveying of trainees who have completed our ATMS program to determine if the skills gained in project management have impacted their research and their transitions into a broad array of careers. Integrating project management skills with research may alleviate some of the pressure from the academic incentive structure that is geared toward productivity (Myers et al., 2023). We also plan to assess faculty perceptions of the impact of ATMS training on the ability of trainees to connect enhanced management of their research projects to research outcomes such as publications, submitted grants, presentations, and student's time to degree. We anticipate that faculty buy-in can create a feedback loop where faculty encourage their trainees to take the course, and the trainees' research outcomes from ATMS reassures faculty of its use, as has been shown with a prior cross-institutional data analysis that evaluates the impact of PhD career and professional development activities on research productivity and efficiency (Brandt et al., 2023).
Interestingly, self-efficacy across the pain points identified in the pretest analysis was statistically lower in graduate students compared to postdocs, a gap that was alleviated after completion of the ATMS training (Figure 3). Project management is included within the organizing and planning subcompetencies of a published competency-based assessment framework for PhD scientists, (Verderame et al., 2018), and demonstratable advancement in this subcompetency throughout training and scientific development is potentially attained in part through assisting and mentoring others. A trainee who has reached the postdoc stage has likely had opportunities to not only manage their own projects, but also oversee the projects of junior trainees and manage project collaborations, which may contribute to their enhanced confidence across the ATMS pain points. In future studies, we plan to survey the types of mentoring and supervisory experiences ATMS participants have had in helping others manage projects in order to determine if these experiences influence initial confidence in navigating project management frameworks.
In future studies, we also plan to explore designing and evaluating curriculum that links project risk management with scientific rigor and reproducibility, an area that is paramount to effective scientific research and a curriculum development priority for funding agencies (Koroshetz et al., 2020). Teaching risk management frameworks to researchers may improve the rigor of research through the structured examination of potential project risks. Additionally, identification and management of risks allows researchers to prepare in advance for events likely to affect their work. This level of thought and control before an experiment may improve the reproducibility of the work by improving the planning and documentation involved.
We have compiled our ATMS curriculum content for dissemination and plan to partner with external colleagues to expand ATMS to trainees in other institutional contexts. As a future study, we plan to assess replicability of ATMS at other institutions. In addition to expanding it to other postdocs and PhD students, we also plan to collaborate with colleagues at Georgetown University and beyond to identify key management competencies for investigator development, and tailor our material for new PIs, or PIs new to project management.
As ATMS trainees graduate or transition to new fields, they offer another partnership opportunity in which alumni can showcase how they use the frameworks in their work. Through these discussions, trainees will be exposed to the many uses of project management as well as the vast field of careers open to them. To that end, we've recently introduced, and will evaluate, a sixth module to ATMS focused on communicating value to new audiences where trainees identify the values, interests, and goals of their audience to tailor the communication of their new project management skills toward broad career options.
6 Study limitations and constraints
A potential limitation of this study is the completion rate of our post-training evaluations (27%, Table 1). A selection bias may be possible where trainees who completed ATMS and the evaluation are the ones who found it most useful. Additionally, this is a single institution study which limits generalizability, and we did not conduct a control group analysis which limits internal validity. However, the results still speak to an overall benefit of the course as the pain points alleviated among the 25 PhD student and postdoctoral trainees who completed the course are represented among the initial pain points identified across all participants who completed the pre-survey (Figures 1, 3). We plan to contact trainees who did not complete ATMS to determine what factors prevented them from finishing, which could include challenges during the COVID-19 pandemic during which the ATMS course was introduced. We also intend to continue to collaborate with university graduate and training programs to incorporate ATMS as a required training component, which could increase completion rates.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The requirement of ethical approval was waived by Georgetown University Institutional Review Board for the studies involving humans because the Georgetown University Institutional Review Board granted an exemption to this study: STUDY00002715, under the following review categories: educational settings; tests, surveys, interviews, or observations (low risk). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants' legal guardians/next of kin because consent is not required for secondary research on data.
Author contributions
ES: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. CM: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare financial support was received for the research, authorship, and/or publication of this article. We would like to thank the Burroughs Wellcome Fund (Grant #1020101) for funding the initial design of the Academy for Transferable Management Skills training program.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Bilal Hysa, E., Akbar, A., Yasmin, F., Rahman, A. U., and Li, S. (2022). Virtual learning during the COVID-19 pandemic: a bibliometric review and future research agenda. Risk Manag. Healthc. Policy 15, 1353–1368. doi: 10.2147/RMHP.S355895
Blomquist, T., Farashah, A. D., and Thomas, J. (2016). Project management self-efficacy as a predictor of project performance: constructing and validating a domain-specific scale. IJPM 34, 1417–1432. doi: 10.1016/j.ijproman.2016.07.010
Brandt, P. D., Whittington, D., Wood, K. D., Holmquist, C., Nogueira, A. T., Gaines, C. H., et al. (2023). Development and assessment of a sustainable PhD internship program supporting diverse biomedical career outcomes. bioRxiv [Preprint]. doi: 10.1101/2023.07.13.548912
Brown, A. M., Meyers, L. C., Varadarajan, J., Ward, N. J., Cartailler, J., Chalkley, R. G., et al. (2023). From goal to outcome: analyzing the progression of biomedical sciences PhD careers in a longitudinal study using an expanded taxonomy. FASEB Bioadv. 5, 427–452. doi: 10.1096/fba.2023-00072
Burroughs Wellcome Fund and Howard Hughes Medical Institute (2006). Making the Right Moves: A Practical Guide to Scientific Management for Postdocs and New Faculty, 2nd Edn. Chevy Chase, MD: Burroughs Wellcome Fund and Howard Hughes Medical Institute.
Chatterjee, D., Ford, J. K., Rojewski, J., and Watts, S. W. (2019). Exploring the impact of formal internships on biomedical graduate and postgraduate careers: an interview study. CBE Life Sci. Educ. 18:ar20. doi: 10.1187/cbe.18-09-0199
Collins, T. R. L., Hoff, K., Starback, M., Brandt, P. D., Holmquist, C. E., and Layton, R. L. (2022). Creating and sustaining collaborative multi-institutional industry site visit programs: a toolkit. F1000Research 9:1317. doi: 10.12688/f1000research.26598.2
Doctoral Recipients from U.S. Universities: 2022 (2023). National Center for Science and Engineering Statistics, Directorate for Social, Behavioral and Economic Sciences. Alexandria, VA: National Science Foundation.
Frank, J. R., Snell, L. S., Cate, O. T., Holmboe, E. S., Carraccio, C., Swing, S. R., et al. (2010). Competency-based medical education: theory to practice. Med. Teach. 32, 638–645. doi: 10.3109/0142159X.2010.501190
Ganapati, S., and Ritchie, T. S. (2021). Professional development and career-preparedness experiences of STEM Ph.D. students: Gaps and avenues for improvement. PLOS ONE 16:e0260328. doi: 10.1371/journal.pone.0260328
Garrison, D. R. (1997). Self-directed learning: toward a comprehensive model. Adult Educ. Q. 48, 18–33. doi: 10.1177/074171369704800103
Kirkpatrick, D., and Kirkpatrick, J. (2006). Evaluating Training Programs: The Four Levels. Oakland, CA: Berrett-Koehler Publishers.
Koroshetz, W. J., Behrman, S., Brame, C. J., Branchaw, J. L., Brown, E. N., Clark, E. A., et al. (2020). Research culture: framework for advancing rigorous research. eLife 9:e55915. doi: 10.7554/eLife.55915
Lee, G. B., and Chiu, A. M. (2022). Assessment and feedback methods in competency-based medical education. Ann. Allergy Asthma Immunol. 128, 256–262. doi: 10.1016/j.anai.2021.12.010
Lenzi, R. N., Korn, S. J., Wallace, M., Desmond, N. L., and Labosky, P. A. (2020). The NIH “BEST” programs: institutional programs, the program evaluation, and early data. FASEB J. 34, 3570–3582. doi: 10.1096/fj.201902064
Mason, J. L., Johnston, E., Berndt, S., Segal, K., Lei, M., and Wiest, J. S. (2016). Labor and skills gap analysis of the biomedical research workforce. FASEB J. 30, 2673–2683. doi: 10.1096/fj.201500067R
Mathur, A., Cano, A., Kohl, M., Muthunayake, N. S., Vaidyanathan, P., Wood, M. E., et al. (2018). Visualization of gender, race, citizenship and academic performance in association with career outcomes of 15-year biomedical doctoral alumni at a public research university. PLOS ONE 13:e0197473. doi: 10.1371/journal.pone.0197473
Muurlink, O., Chen, L., Boorman, R., Pearson, D., and Cohen, G. (2024). Stakeholder perceptions of what industry wants from doctoral students: a systematic literature review. High. Educ. Res. Dev. 43, 952–965. doi: 10.1080/07294360.2023.2269871
Myers, K. R., Tham, W. Y., Thursby, J., Thursby, M., Cohodes, N., Lakhani, K., et al. (2023). New facts and data about professors and their research. arXiv. doi: 10.48550/arXiv.2312.01442
NOT-OD-20-031: Notice of NIH's Interest in Diversity (2019). Available at: https://grants.nih.gov/grants/guide/notice-files/NOT-OD-20-031.html (accessed July 30, 2024).
Project Management Institute (2017). A Guide to the Project Management Body of Knowledge (PMBOK Guide), 6th Edn. Project management Institute.
Ramadoss, D., Bolgioni, A. F., Layton, R. L., Alder, J., Lundsteen, N., Stayart, C. A., et al. (2022). Using stakeholder insights to enhance engagement in PhD professional development. PLOS ONE 17:e0262191. doi: 10.1371/journal.pone.0262191
Sinche, M., Layton, R. L., Brandt, P. D., O'Connell, A. B., Hall, J. D., Freeman, A. M., et al. (2017). An evidence-based evaluation of transferrable skills and job satisfaction for science PhDs. PLOS ONE 12, e0185023. doi: 10.1371/journal.pone.0185023
Thomas, P. A., David, E. K., Hughes, M. T., Tackett, S. A., and Chen, B. Y. (2022). Curriculum Development for Medical Education. Baltimore, MD: Johns Hopkins University Press. doi: 10.56021/9781421444116
Van Wart, A., O'Brien, T. C., Varvayanis, S., Alder, J., Greenier, J., Layton, R. L., et al. (2020). Applying experiential learning to career development training for biomedical graduate students and postdocs: perspectives on program development and design. CBE Life Sci. Educ. 19:es7. doi: 10.1187/cbe.19-12-0270
Verderame, M. F., Freedman, V. H., Kozlowski, L. M., and McCormack, W. T. (2018). Competency-based assessment for the training of PhD students and early-career scientists. Elife 7:e34801. doi: 10.7554/eLife.34801
Waidelich, L., Richter, A., Kölmel, B., and Bulander, R. (2018). “Design thinking process model review,” in 2018 IEEE International Conference on Engineering, Technology and Innovation (ICE/ITMC) (Stuttgart: IEEE), 1–9. doi: 10.1109/ICE.2018.8436281
Watts, S. W., Chatterjee, D., Rojewski, J. W., Reiss, C. S., Baas, T., Gould, K. L., et al. (2019). Faculty perceptions and knowledge of career development of trainees in biomedical science: what do we (think we) know? PLOS ONE 14:e0210189. doi: 10.1371/journal.pone.0210189
Keywords: project management, transferable skills, experiential learning, design thinking, skills development, career development, professional development
Citation: Salm EJ and McKinney CC (2024) Design and implementation of a project management training program to develop workforce ready skills and career readiness in STEM PhD students and postdoctoral trainees. Front. Educ. 9:1473774. doi: 10.3389/feduc.2024.1473774
Received: 31 July 2024; Accepted: 07 October 2024;
Published: 25 October 2024.
Edited by:
Adriana Bankston, AAAS STPF/AGCT Congressional Policy Fellow, United StatesReviewed by:
Susi Sturzenegger Varvayanis, Cornell University, United StatesJacqueline McLaughlin, University of North Carolina at Chapel Hill, United States
Constance Stayart, The University of Chicago, United States
Copyright © 2024 Salm and McKinney. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Caleb C. McKinney, Y2FsZWIubWNraW5uZXkmI3gwMDA0MDtnZW9yZ2V0b3duLmVkdQ==