Emergency residents' self-perceived readiness for practice: the association of milestones, entrustable professional activities, and professional identities—a multi-institutional survey

Background: As a successful innovation, competency-based medical education and its assessment tools continue to be a key strategy in training future doctors and tracking their performance trajectories. Linked to professional identity, evidence suggests that clinical competence is related to thinking, acting and feeling like a physician. Thus, incorporating the values and attitudes of healthcare professions as part of their professional identity in the clinical workplace improves professional performance.

Methods: Through a cross-sectional study, we examined the association of milestone, entrustable professional activities (EPA) and professional identity using self-reported tools among emergency medicine residents from 12 teaching hospitals across Taiwan. Milestone, EPA and professional identity were assessed using the Emergency Medicine Milestone Scale, Entrustable Professional Activity Scale and Emergency Physician Professional Identity and Value Scale, respectively.

Results: The results of a Pearson correlation indicated a significant positive correlation between milestone-based core competencies and EPAs (r = 0.40 ~ 0.74, p < 0.01). The professional identity domain of skills acquisition, capabilities and practical wisdom was positively correlated with milestone-based core competencies of patient care, medical knowledge, practice-based learning and improvement, and system-based practice (r = 0.18 ~ 0.21, p ≤ 0.05), and six items of EPA (r = 0.16 ~ 0.22, p < 0.05). Additionally, the professional identity domain of professional recognition and self-esteem was positively correlated with practice-based learning and improvement, and system-based practice milestone competencies (r = 0.16 ~ 0.19, p < 0.05).

Conclusion: This study demonstrates milestone and EPA assessment tools are highly linked and therefore, can be synergistically used by supervisors and clinical educators to evaluate clinical performance during residency training. Emergency physicians’ professional identity is partly influenced by the advancement of skills and a resident’s ability to learn, effectively perform tasks and make appropriate medical decisions at the system level in their clinical practice. Further research is warranted to understand the importance of residents’ competency in relation to their professional identity development trajectory during clinical training.

Introduction

The implementation of competency-based medical education (CBME) to help medical trainees to smoothly transition into the clinical workplace and as a scaffold for readiness to practice is valuable (1, 2). Nonetheless, nowadays, it is anticipated for medical training programs to concurrently promote clinical competence and professional identity formation in clinical practice (3). This is because thinking, acting and feeling like a physician is key to professional development (4). Despite this proposed significance, the relationship between competence (doing) and professional identity (being or feeling like a physician) remains ambiguous. In this study, we argue that central to learning in preparation for readiness to practice, is the gradual participation of junior medical trainees within their community of practice through peripheral legitimate patient care under supervision to independent decision making in medical practice which enhances their sense of membership and professionalism (4, 5). This is consistent with Cruess et al. (4) conceptualization of the non-linear and complicated relationship between “doing” and “being” stages of practice. The conceptualization depicts how social interactions over time facilitate both the accumulating experiences and the internalization of professional attitudes, values and beliefs, necessary to help medical trainees transition into clinical practice. Consequently, the construction of professional identity represents the final stage in the trajectory of professional development; helping medical trainees to build strong relationships with their co-workers, belongingness in their community of practice which promote confidence in themselves, and a better sense of independence (5–7). Drawing from Cruess et al. (4) amended framework of Miller’s pyramid, this study aims to investigate the association between “doing” and “being” a healthcare professional.

By examining this relationship, our study serves as a response to Cruess’s call to assess the association between competencies and professional identities among healthcare professionals. Both competencies and professional identity play important roles in the socialization process of medical trainees and their level of professionalism; a form of social contract with the public to deliver quality care that all physicians commit to uphold (8). Therefore, the results of our study sheds light on the professional development of the physician workforce, which the public expects to see.

Background

Miller’s pyramid, a broadly utilized framework for assessing medical trainees competence, describes four levels of knowledge and skills: Knows, Knows how, Shows how and Does (9). Through the lens of this pyramid, medical trainees’ competence and performance can be assessed from foundational medical knowledge (Knows) towards functioning independently in clinical situations (Does). Despite the framework receiving tremendous support in the educational setting as a very useful tool in facilitating teaching and learning (10), it has been criticized for its static nature which impedes it from reflecting performance in the real workplace, where dynamic factors are at play (4). Therefore, Cruess et al. (4) amended Miller’s pyramid includes the fifth level “Is” at its apex depicting professional identity formation. Thus, incorporating the process of developing values, attitudes and beliefs of healthcare professionals, together with the process of developing their competencies is of clinical importance. Consequently, this new Cruess et al. (4) framework proposes that the learning outcome in the workplace can represent a medical trainee’s professional development trajectory moving from “doing” to “being.”

Professional identity

According to Cruess et al. (4), the state of “being” and “feeling” like a physician are the hallmark of professional identity. Indeed, a growing body of research drawing from social constructionism, suggests that professional identities constitutes of attitudes, professional codes, beliefs and values that are characteristics of a profession, internalized by members through ongoing interactions and dynamic co-construction of their work experiences (11, 12). Likewise, building on this dynamic conceptualization of professional identities, Chang et al. (13) documented the complexity of emergency physicians’ work scope and work environment with its implications on the construction of their professional identities. Uncovering five key conceptual metaphors; safety net, gateway, market, sports and war that emergency physicians use to ascribe the value of and meaning to emergency care, the authors concluded that the metaphors give insights into the scaffolds that emergency physicians use to construct their multifaceted professional identities (13).

In a subsequent study which used a different sample, Chang et al. (14) teased out the dominating perspectives that emergency physicians held about their professional identities by analyzing the priorities they place on their previously reported roles, ethical codes, standards, values and beliefs. The results indicated four main viewpoints which underpin emergency physician’s professional identities. Viewpoint 1- skills acquisition, capabilities and practical wisdom; Viewpoint 2- coping ability and resilience; Viewpoint-3: professional recognition and self-esteem; and Viewpoint 4: wellbeing and quality of life (14). Viewpoint 1 highlighted having competence necessary to handle the challenges in emergency daily practice and the unpredictable working environment distinguished emergency specialties from other healthcare professions. Viewpoint 2 focused on career prospects and prioritized the importance personal physical strength and department leadership by which emergency physician can work under pressure and in charge of sudden events that arise at work. Viewpoint 3 accentuated being recognized within the institution and by the larger public can support the role emergency physicians play in the larger social context. Viewpoint 4 spotlighted to prioritize non-work commitments such as family and personal amusement and maintain the work-life balance can mitigate work pressure and contribute their success at work. Overall, this conceptualization of professional identity underpinned by context dependent co-construction and negotiation of values and beliefs lends itself to the operationalization of the professional identities of emergency physicians, a group of healthcare professions with multidisciplinary expertise whose work encompasses providing undifferentiated care to patients with acute care needs. It also echoes with Cruess’s conceptualization of the complex relationship between doing” and “being” a physician shaped by the interplay between the learning environment, individual identities, social interactivities and experiences.

Competency-based assessments

CBME as a successful innovation continues to be a key strategy in educating future doctors and other healthcare professionals (15, 16). Conducting valid and robust assessments in graduate medical education is essential for effective feedback and prepares the learner’s readiness for practice in real and complex clinical settings (2, 17). Thus, well-grounded assessment is a social responsibility among healthcare professionals to ensure quality of care is provided by the medical community. To date, such assessments include milestone assessments and entrustable professional activities.

The Accreditation Council for Graduate Medical Education (ACGME) milestones guidebook has highlighted that residency training programs need to emphasize the top level of the Miller’s pyramid “Does” which refers to work-based assessments (18). Milestones are educational outcomes and observable performance markers for residents professional development (19). Six core-competency of milestone assessment namely patient care and procedural skills, medical knowledge and clinical reasoning, professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice have been recommended by the ACGME (18). Evidence shows the Miller’s Pyramid assessment outcome and the ACGME six core-competencies are significantly related (10).

Likewise, entrustable professional activities (EPAs) which are independent executable key tasks residents need to perform in their respective medical specialties, are fundamental to a profession and entrusted to a trainee to complete without supervision (20, 21). Entrustment decision implies clinical educators accept the inherent risks related to the clinical task and their judgement of the level of supervision needed for medical trainees to deliver good quality of care. Thus, EPAs are beyond the observation of the “Does” level of practice in Miller’s pyramid.

For residents, milestones are educational outcomes and observable performance markers in their professional development (19), while EPAs are independent executable key tasks residents need to perform in their respective medical specialties. The integrated frameworks of milestones and EPAs are expected to define and determine progression trajectory of individuals’ clinical competencies in the workplace (20, 22). Through the strategy of mapping, evidence has shown milestones are closely linked to EPAs by creating a unified structure that prepares residents to complete specified tasks effectively and autonomously in clinical practice (20, 22–24). However, despite the availability of such method, empirically there still exists the gap to confirm the link between milestone competency and EPA assessment across various medical specialties. Previous studies indicate a potential association between the two assessments (25, 26). Based on the empirical evidence, we hypothesize that milestone assessment is related to EPA assessment outcomes. Knowing such relationship in this study provides supervisors with a shared mental model of performance expectations for trainees (27, 28), reduces undesirable variability and improves assessment accuracy by giving specific feedback with regards to a resident’s professional development (29).

Professional identity and competency-based assessments

Using the Miller’s pyramid (9), and Cruess’s amended pyramid as a guide in this study (4), we argue that continual understanding of the learning outcomes’ progression, and the relationships between competency-based assessment and professional identity measure is paramount. Nevertheless, few studies have investigated the relationship of competency-based assessment on professional identities among healthcare professions (30, 31). To add to the literature, our study examined how emergency medical residents (EMRs) self-assess their clinical performance for quality of patient care by using milestone-based competencies and EPA assessment tools, and how these assessments relate to how EMRs perceive themselves as emergency physicians.

Therefore, we hypothesized that milestone-based core competencies and EPA assessment tools will positively correlate with professional identity. The understanding of the association provides evaluative judgement of task-specific clinical skills and the professional identity residents possess to stakeholders of CBME medical residency training programs.

Methods

Design and setting

A cross-sectional study of EMRs was conducted between April and July 2021 at 12 tertiary teaching hospitals across Taiwan with Residency Review Committee (RRC) approved emergency medicine residency programs. This study is part of a multi-year research regarding the incorporation of CBME and its subsequent assessment methods in emergency medicine residency training programs in Taiwan. Recently, Taiwan has been recognized as an international hub for ACGME regional faculty development (32).

Participants

EMRs were invited through emails and posters pasted in the emergency department offices. Program directors of 46 RRC approved emergency residency training programs were further contacted to connect the research team with interested participants. Out of the 46 RRC approved programs, only those that had an ongoing implemented milestone and EPA assessment protocol were included as the final sample recruitment. EMRs of all residency years were recruited using a convenience sampling. Written informed consents were obtained from participants. The consent forms were collected directly if participants had the time to complete the questionnaire at the study site. If not, they were mailed back to the investigators. Participation was voluntary and anonymity was guaranteed through the use of identification numbers. The study was approved by the Chang Gung Medical Foundation Institutional Review Board (No: 201900082B0C601).

Data collection and instruments

An online self-reported questionnaire using Google link was used for data collection and required approximately 20 min to complete. Considering this study was part of a multi-year project until July 2022, participants completed the questionnaire at their convenience. However, a follow-up was initiated to participants who delayed. The questionnaire included:

Milestones-based ACGME core competencies

The Chinese version of the Emergency Medicine Milestone (EMM) developed in 2014 by the Taiwan Society of Emergency Medicine (TSEM) using standardized consensus methods was used for assessment (33). In 2018, a revised version of this milestone assessment tool was introduced for the training of EMRs in Taiwan (34). EMM measures the stages that mark the development and acquisition of residents’ competencies using 23 sub-competency items which are aggregated into the ACGME six-core competencies. These include patient care (PC, 14 items), medical knowledge (MK, 1 item), professionalism (PROF, 2 items), interpersonal and communication skills (ICS, 2 items), practice-based learning and improvement (PBLI, 1 item), and system-based practice (SBP, 3 items) (34–36). A detailed list of the ACGME six-core competencies and their associated 23 sub-competencies are provided in supplementary file (Supplementary Table S1). The 23 sub-competency items are measured on a 5-level scale of proficiency with a higher level indicating higher competency (34, 35). Specifically, level 4 has been designed as the graduation target for residents (33).

Entrustable professional activities

EPAs were assessed using the Chinese version of the EPA scale developed by the TSEM (37). The Chinese EPA scale contains seven entrustable activities and each item is measured on a 5-level of proficiency with a higher level representing a higher competency (37). The seven items include: item 1—Management of out-of-hospital cardiac arrest patients; item 2—Management of shock patients; item 3—Management of patients with major trauma; item 4—Management of poisoned patients; item 5—Management of patients with acute chest pain; item 6—Management of patients with acute consciousness change; and item 7—Management of patients with acute dyspnea (37).

Professional identity

The Chinese version of the Emergency Physician Professional Identity and Value Scale (EPPIVS) was used to measure PI (Supplementary Table S2). The EPPIVS is a 20-item, self-reported questionnaire on a 7-point Likert scale. Underpinned by a construction of theoretical frameworks, the questionnaire was developed by a team of emergency physicians and other professionals in Taiwan (14). The 20 items measure four domains of professional identity: (1) skills acquisition, capabilities, and practical wisdom—7 items, (2) coping ability and resilience—4 items, (3) professional recognition and self-esteem—5 items, and (4) well-being and quality of life—4 items (14). With 150 participants, the initial scale reported a Cronbach’s α of ≥0.76 for all domains amongst emergency physicians (EPs) and EMRs in a single healthcare system. Evidence shows professional identity is multi-faceted or multi-domain and influenced by both internal and external factors (11, 14, 38, 39).

Demographic characteristics

The demographic characteristics measured included age, gender, marital status, highest degree attained, current residency year, hours worked/month, frequency of milestone and EPA evaluation within a 3-month training span, clinical, teaching and research workload, and number of residents in the program and in the same year of residency training.

Data analysis

Analysis was performed using SPSS 24 (SPSS, Chicago, IL, United States). Descriptive statistics including frequency with percentage and mean with standard deviation (SD) were used to summarize participants’ characteristics after testing for normality. Bivariate analysis using chi-square test for categorical variables or a one-way analysis of variance (ANOVA) for continuous variables was performed to compare participants’ demographics and their residency years (40). Pearson correlation test was used to examine the relationship between the competency-based assessments—milestone and EPA, and professional identity. For better interpretation, the results of the 23 sub-competency milestone assessments were aggregated to form the ACGME six core competency results (18). Correlations (r) ranging 0.10 ~ 0.29 were categorized as small, 0.30 ~ 0.49 as medium and 0.50 ~ 1.00 as large (41). A value of p of ≤0.05 was considered as significant.

Results

Participant characteristics

The mean age of participants was 30.8 ± 2.7 years (Table 1). Of the 171 EMRs, 131 (76.6%) were male, 127 (74.3%) single, 164 (95.9%) had a bachelor’s degree and 115 (67.3%) worked >180 h/month. The participants were equally distributed across their residency years with the average of 4.4 ± 2.6 belonging to the same class. Overall, the average number of residents in-training was 16.9 ± 9.7. The mean reported percentage of workload for clinical, teaching and research was 82.0 ± 14.3, 12.7 ± 8.8, and 5.0 ± 4.8, respectively. Likewise, the mean number of milestone and EPA assessments conducted among the participants within the past 3 months were 9.3 ± 16.6 and 7.9 ± 12.5, respectively. The chi-square test or ANOVA showed no statistically significant differences (p = 0.10 ~ 0.99), except for age when comparing participants’ characteristics with their residency years of practice.

TABLE 1

Table 1. Participant demographics across residency years (N = 171).

Correlation of competency-based assessments

To assess the relationship between the ACGME milestone-based core competencies and EPA, a correlation analysis was performed. A significant positive correlation was demonstrated between the ACGME six milestone-based core competencies and the 7 EPA items (r = 0.40 ~ 0.74, p < 0.01) (Table 2). In Figures 1, 2, the radar plots illustrate the profiles of self-assessed milestones and EPAs with residency years. The results show residents self-assessment of milestone and EPA scores significantly increased with residency years.

TABLE 2

Table 2. Correlation of milestone-based competencies and EPA assessment.

FIGURE 1

Figure 1. Radar plots of residency years with milestone-based competency assessment.

FIGURE 2

Figure 2. Radar plots of residency years with EPA assessment.

Correlation of professional identity and competency-based assessments

The professional identity questionnaire used in this study reported a Cronbach’s α of ≥0.70 for the subscales. However, the scores of the four dimensions of professional identities remained constant against residency years. The correlation between professional identity subscales with milestone-based core competencies and EPA showed a positive relationship (Table 3). The skills acquisition, capabilities, and practical wisdom domain of professional identities was positively correlated with 4 ACGME milestone-based core competencies including patient care, medical knowledge, practice-based learning and improvement, and system-based practice (r = 0.18 ~ 0.21, p ≤ 0.05) and 6 items of EPA (r = 0.16 ~ 0.22, p < 0.05). Additionally, the professional recognition and self-esteem domain of professional identities was positively correlated with 2 ACGME milestone-based core competencies including practice-based learning and improvement and system-based practice (r = 0.16 ~ 0.19, p < 0.05).

TABLE 3

Table 3. Correlation of professional identity with milestone-based competencies and EPA assessment.

Discussion

This multi-institutional survey is the first study to explore on EMRs self-evaluative judgement of their clinical skills trajectory using the ACGME milestone-based core competency and EPA assessment, and further, examine the association between task-specific clinical skills and professional identity. Using the amended Miller’s pyramid competency framework suggested by Cruess et al. (4), our study outcome advances the understanding of how EMRs perceive their readiness for practice by assessing the quality of “doing” and “being” an emergency physician in the emergency workplace, a unique clinical context under the social contract between emergency medicine and the public. Based on EMRs self-evaluation of their clinical skills, a significant positive relationship was observed between the ACGME six milestone-based core competencies and EPA assessment. Furthermore, the EMRs competencies indicate progression during the residency training years. However, their multi-faceted professional identities remained static. Only one dimension of professional identity—skills acquisition, capabilities, and practical wisdom, was weakly correlated to most milestone and EPA subscales or items. Our study, therefore, is an innovative implementation research bringing more understanding in regards to the intercorrelations between the ACGME milestone core competencies, EPA and professional identity to the stakeholders of CBME residency training programs.

The goal of medical education is to build a healthcare workforce that is able to provide high-quality care without requiring active supervision (29). Milestone alone provides more timely and specific assessment of healthcare provider performance and may enhance existing biannual competency-based formative feedback tools (25). Furthermore, connecting ACGME milestone reports to EPA-based evaluations allow training programs to use the competency assessment tools most related to their clinical context and assessors’ construct (20, 42, 43), while still apprehending broader competency progression for decisions of promotion and reporting to accrediting bodies (19, 23, 24, 44). With limited studies conducted on the association of milestone-based competence and EPA assessment, our finding adds to the empirical reliability and validity of competency-based assessments. Additionally, it shows that milestone and EPA assessments can be synergistically used in the workplace when evaluating EMRs (25). As expected, there exists a relationship between milestone and EPA as they both define residents’ performance trajectories (22). This finding is similar to previous studies that found a moderate to high positive correlation between milestones and EPA assessment in pediatric, internal medicine and surgical residency training, which is innovative for emergency medicine (25, 26, 45) Through our survey, milestones and EPA assessment were, therefore, designed to define the tasks and standards that can lead to entrustability and autonomy of EMRs (20). This can further provide a foundation towards developing effective educational interventions that increases residents’ performance in the clinical practice.

Evaluative judgement can be defined as the ability to determine the quality of work that has been completed by an individual and others in a specific context (46, 47). It reflects on the long-term impacts of assessment on the professional development of trainees and thereby, contributing to residents’ construct of their professional identity (48). It is evident from literature that knowledge, attitude and clinical skills acquisition are intercorrelated (49), which are important aspects that can further reinforce residents’ professional identity (3, 14). In the present study, the skills acquisition, capabilities, and practical wisdom domain of professional identity was weakly but comprehensively correlated to most competency-based assessments, which highlighted EMRs capabilities to multi-task, diagnose a medical condition quickly and experience the fresh challenges that the emergency department brings every day. In turn, this helps to make sense of their learning on how to act and behave like an emergency physician from co-workers in the emergency setting (14). Additionally, the professional recognition and self-esteem domain of professional identity was correlated to two milestone-based core competency subscales—practice-based learning and improvement, and system-based practice, showing EMRs ability to utilize systematic efforts to improve quality, safety or value of healthcare services, and to conduct interprofessional care and practice in a complex healthcare system necessary for their professional identity formation (14). Thus, the development of skills and a resident’s ability to learn, competently perform tasks and make sound medical-related decisions for quality of patient care at a larger context and system is of importance for EMRs and partly influences their professional identity (49). Evident from the lack of correlation found in this study between competency-based assessments and professional identity, one possible reason is that emergency medicine professional identities are multifaceted, constructed and impacted by experiencing different levels of sociocultural identification (11). We therefore, suggest the need to further develop and validate EMRs competency-based assessment tools to capture all the important aspects that affect physicians’ professional identity development.

Given that one fundamental aspect of community of practice is that members should have mutual engagement in tasks and shared goals, milestone-based competency and EPA assessments have been designed and implemented under consensus. This can be used to identify activities and competencies unique to a particular profession or specialty such as the emergency department (50). By being an active participant, competently performing the specified competencies, and reflecting on their progress, trainees’ professional identities are expected to form and develop over time (51). However, EMRs professional identity domains in this study remained static contrary to the task-specific competencies, which evolved with residency years. Considering our study participants were all residents in training in various residency years who provided medical care under supervision in different emergency institutions, their professional identity is in infancy and still developing. This is consistent with Cruess’s amended model of competence which considers the professional identity as the last stage of competence (4). Hence, the EMRs professional identities will continue to progress and be clearer as the gain additional work experiences either as an attending physician or clinical educator. Through a social identity lens, the construction of residents’ identity is a two way process of maintaining existing identities and gaining new identities toward social identity compatibility, which takes time to be developed and tested (52). Our study further adds knowledge into the emergency medicine community of practice in that milestone-based competency and EPA assessments are not major determining factors of professional identity.

The practical implications of our study come from the establishment of the relationship between milestone and EPA assessment in the clinical setting which facilitates effective clinical supervision for residents in various specialties. We suggest that milestones can be used in conjunction with EPA assessment. Exploring the association informs attending physicians or clinical educators to complement milestone and EPAs during assessment for a holistic evaluation. Instead of using the assessment tools separately, simultaneous use achieves a comprehensive and rigorous assessment. Likewise, using the competency assessment tools together allows residents to be informed of their competency trajectory in a reliable and comprehensive manner. This can increase their ability to make sense of their professional development trajectory; promoting their receptivity to feedback.

Strengths and limitation

The strengths of our study lie in the recruitment of participants from various teaching hospitals in Taiwan that have RRC qualified emergency medicine residency training programs, which adds to the generalizability of the results. We also utilized the self-reported assessments following the scaffold recommended by the ACGME, which plays an important role for EMRs self-evaluative judgement to define their quality of work and future professional development. Likewise, through the lens of extending the Miller’s pyramid, we add to the gap and shed light on how to prepare medical training programs to concurrently promote clinical competence and professional identity development in the clinical workplace. However, our study is not without limitations, and careful considerations should be made when interpreting and applying the findings to other contexts. Possibly only 12 out of 46 RRC qualified training programs have fully implemented their CBME at a larger scale were enrolled in this study. However, this was a multi-institutional survey which recruited 171 EMRs across Taiwan and accounted for 40% of total training capacity in the 46 residency programs. Our study results therefore, can still be generalized to the residency population. We used a cross-sectional study to explore the relationship between milestone-based competency and EPA assessment, and professional identity, which was unable to show the cause-effect relationships. Future studies to consider longitudinal study designs and examine such cause-effect relationships by collecting and analyzing the objective workplace assessment data is needed. Despite these limitations, our study is the first to assess and provide a basis for understanding the link between competency-based assessment tools and professional identity among EMRs from an East Asian perspective.

Conclusion

This study highlights the intercorrelation between milestone-based competency and EPA assessments with multifaceted professional identities among EMRs. The implementation of assessment methods for evaluating progress of medical trainees and professional identity is needed to assess their readiness for practice. A highly positive correlation existed between the competency-based assessment tools—milestone and EPA—indicating they can be synergistically used by supervisors and clinical educators to evaluate clinical performance during residency training. Despite theoretical evidence showing competence and professional identity go hand in hand, our study found that at the residency stage the association between task-specific competencies and professional identities is partial. This suggests that in addition to focusing on the acquisition of skills, EMRs still need strong support from their communities of practice to facilitate and reinforce the development of their professional identities during training. Therefore, further research is needed to understand how resident’s competency trajectory aligns with how they internalize different values that underpin their professional identities during clinical training.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Ethics statement

Ethical approval was obtained from the Chang Gung Medical Foundation Institutional Review Board (No: 201900082B0C601). Participants provided their written informed consent before participation.

Author contributions

Y-CC and NN: conceptualization. Y-CC, NN, MK, and RC: methodology. MK, Y-CC, and C-HC: formal analysis and investigation. MK: writing—original draft preparation. Y-CC, MK, RC, and C-HC: writing—review and editing. Y-CC: supervision. All authors contributed to the article and approved the submitted version.

Funding

This study was funded by Ministry of Science and Technology, Taiwan (Grant no. 108-2511-H-182-007-MY2).

Acknowledgments

Our appreciation goes to all the EMRs who took part in this study. The authors are also thankful to Fidelia Berenice Prayugo, Fremen Chihchen Chou, Chih-Wei Yang, Hsu-Min Tseng, Chia-Te Kung, Shu-Chen Liao, and Lynn Monrouxe for their contributions and making this work a success.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2023.1032516/full#supplementary-material

References

1. Bleakley, A, and Brennan, N. Does undergraduate curriculum design make a difference to readiness to practice as a junior doctor? Med Teach. (2011) 33:459–67. doi: 10.3109/0142159X.2010.540267

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Li, S-TT, Tancredi, DJ, Schwartz, A, Guillot, AP, Burke, AE, Trimm, RF, et al. Competent for unsupervised practice: use of pediatric residency training milestones to assess readiness. Acad Med. (2017) 92:385–93. doi: 10.1097/ACM.0000000000001322

CrossRef Full Text | Google Scholar

3. Parsons, AS, Kon, RH, Plews-Ogan, M, and Gusic, ME. You can have both: coaching to promote clinical competency and professional identity formation. Perspect Med Educ. (2021) 10:57–63. doi: 10.1007/s40037-020-00612-1

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Cruess, RL, Cruess, SR, and Steinert, Y. Amending Miller's pyramid to include professional identity formation. Acad Med. (2016) 91:180–5. doi: 10.1097/ACM.0000000000000913

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Rees, CE, and Monrouxe, LV. Who are you and who do you want to be? Key considerations in developing professional identities in medicine. Med J Aust. (2018) 209:202–3. doi: 10.5694/mja18.00118

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Hay, A, Smithson, S, Mann, K, and Dornan, T. Medical students' reactions to an experience-based learning model of clinical education. Perspect Med Educ. (2013) 2:58–71. doi: 10.1007/S40037-013-0061-4

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Sawatsky, AP, Santivasi, WL, Nordhues, HC, Vaa, BE, Ratelle, JT, Beckman, TJ, et al. Autonomy and professional identity formation in residency training: a qualitative study. Med Educ. (2020) 54:616–27. doi: 10.1111/medu.14073

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cruess, RL, and Cruess, SR. Professionalism, communities of practice, and medicine's social contract. J Am Board Fam Med. (2020) 33:S50–6. doi: 10.3122/jabfm.2020.S1.190417

CrossRef Full Text | Google Scholar

9. Miller, GE. The assessment of clinical skills/competence/performance. Acad Med. (1990) 65:S63–7. doi: 10.1097/00001888-199009000-00045

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Williams, BW, Byrne, PD, Welindt, D, and Williams, MV. Miller's pyramid and core competency assessment: a study in relationship construct validity. J Contin Educ Health Prof. (2016) 36:295–9. doi: 10.1097/CEH.0000000000000117

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Monrouxe, LV. Identity, identification and medical education: why should we care? Med Educ. (2010) 44:40–9. doi: 10.1111/j.1365-2923.2009.03440.x

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Rees, CE, Kent, F, and Crampton, PE. Student and clinician identities: how are identities constructed in interprofessional narratives? Med Educ. (2019) 53:808–23. doi: 10.1111/medu.13886

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Chang, Y-C, Nkambule, NS, Xiao, X, Ngerng, RY, and Monrouxe, LV. Safety net, gateway, market, sport, and war: exploring how emergency physicians conceptualize and ascribe meaning to emergency care. Soc Sci Med. (2021) 287:114338. doi: 10.1016/j.socscimed.2021.114338

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Chang, Y-C, Xiao, X, Nkambule, N, Ngerng, RY, Bullock, A, and Monrouxe, LV. Exploring emergency physicians' professional identities: a Q-method study. Adv Health Sci Educ. (2021) 26:117–38. doi: 10.1007/s10459-020-09973-y

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hawkins, RE, Welcher, CM, Holmboe, ES, Kirk, LM, Norcini, JJ, Simons, KB, et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ. (2015) 49:1086–102. doi: 10.1111/medu.12831

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Mulder, H, Cate, OT, Daalder, R, and Berkvens, J. Building a competency-based workplace curriculum around entrustable professional activities: the case of physician assistant training. Med Teach. (2010) 32:e453–9. doi: 10.3109/0142159X.2010.513719

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Levett-Jones, T, Gersbach, J, Arthur, C, and Roche, J. Implementing a clinical competency assessment model that promotes critical reflection and ensures nursing graduates' readiness for professional practice. Nurse Educ Pract. (2011) 11:64–9. doi: 10.1016/j.nepr.2010.07.004

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Eno, C, Correa, R, Stewart, N, Lim, J, Westerman, ME, Holmboe, ES, et al. Milestones Guidebook for Residents and Fellows. Chicago: Accreditation Council for Graduate Medical Education (Acgme) (2020). 19 p.

Google Scholar

19. Swing, SR, Beeson, MS, Carraccio, C, Coburn, M, Iobst, W, Selden, NR, et al. Educational milestone development in the first 7 specialties to enter the next accreditation system. J Grad Med Educ. (2013) 5:98–106. doi: 10.4300/JGME-05-01-33

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hart, D, Franzen, D, Beeson, M, Bhat, R, Kulkarni, M, Thibodeau, L, et al. Integration of entrustable professional activities with the milestones for emergency medicine residents. West J Emerg Med. (2019) 20:35–42. doi: 10.5811/westjem.2018.11.38912

CrossRef Full Text | Google Scholar

21. Ten Cate, O, Chen, HC, Hoff, RG, Peters, H, Bok, H, and Van Der Schaaf, M. Curriculum development for the workplace using entrustable professional activities (Epas): Amee guide no. 99. Med Teach. (2015) 37:983–1002. doi: 10.3109/0142159X.2015.1060308

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Carraccio, C, Englander, R, Gilhooly, J, Mink, R, Hofkosh, D, Barone, MA, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. (2017) 92:324–30. doi: 10.1097/ACM.0000000000001141

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Choe, JH, Knight, CL, Stiling, R, Corning, K, Lock, K, and Steinberg, KP. Shortening the miles to the milestones: connecting Epa-based evaluations to Acgme milestone reports for internal medicine residency programs. Acad Med. (2016) 91:943–50. doi: 10.1097/ACM.0000000000001161

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Li, S-TT. The promise of milestones: are they living up to our expectations? J Grad Med Educ. (2017) 9:54–7. doi: 10.4300/JGME-D-16-00694.1

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Albright, JB, Meier, AH, Ruangvoravat, L, and Vandermeer, TJ. Association between entrustable professional activities and milestones evaluations: real-time assessments correlate with semiannual reviews. J Surg Educ. (2020) 77:e220–8. doi: 10.1016/j.jsurg.2020.07.027

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Larrabee, JG, Agrawal, D, Trimm, F, and Ottolini, M. Entrustable professional activities: correlation of entrustment assessments of pediatric residents with concurrent subcompetency milestones ratings. J Grad Med Educ. (2020) 12:66–73. doi: 10.4300/JGME-D-19-00408.1

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Edgar, L, Jones, MD Jr, Harsy, B, Passiment, M, and Hauer, KE. Better decision-making: shared mental models and the clinical competency committee. J Grad Med Educ. (2021) 13:51–8. doi: 10.4300/JGME-D-20-00850.1

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Scheurer, JM, Davey, C, Pereira, AG, and Olson, AP. Building a shared mental model of competence across the continuum: trainee perceptions of subinternships for residency preparation. J Med Educ Curric Dev. (2021) 8:23821205211063350. doi: 10.1177/23821205211063350

CrossRef Full Text | Google Scholar

29. Carraccio, C, Englander, R, Holmboe, ES, and Kogan, JR. Driving care quality: aligning trainee assessment and supervision through practical application of entrustable professional activities, competencies, and milestones. Acad Med. (2016) 91:199–203. doi: 10.1097/ACM.0000000000000985

CrossRef Full Text | Google Scholar

30. Haghighat, S, Borhani, F, and Ranjbar, H. Is there a relationship between moral competencies and the formation of professional identity among nursing students? BMC Nurs. (2020) 19:49. doi: 10.1186/s12912-020-00440-y

CrossRef Full Text | Google Scholar

31. Mohtashami, J, Rahnama, H, Farzinfard, F, Talebi, A, Atashzadeh-Shoorideh, F, and Ghalenoee, M. A survey of correlation between professional identity and clinical competency of psychiatric nurses. Open J Nurs. (2015) 05:765–72. doi: 10.4236/ojn.2015.59080

CrossRef Full Text | Google Scholar

32. Iobst, WF, and Holmboe, ES. Programmatic assessment: the Secret sauce of effective CBME ImplementationJ Grad. Med. Educ. (2020) 12:518–521. doi: 10.4300/JGME-D-20-00702.1

CrossRef Full Text | Google Scholar

33. Yang, C-W, Hsiao, C-T, and Chou, FC. Emergency medicine milestone project for residency training in Taiwan. 醫學教育. (2017) 21:73–80.

Google Scholar

34. Taiwan Society Of Emergency Medicine. (2018). The emergency medicine milestone project [online]. Available at: https://www.sem.org.tw/ (Accessed October.22, 2021).

Google Scholar

35. Ling, LJ, and Beeson, MS. Milestones in emergency medicine. JACME. (2012) 2:65–9. doi: 10.1016/j.jacme.2012.06.002

CrossRef Full Text | Google Scholar

36. Ten Cate, O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. (2013) 5:157–8. doi: 10.4300/JGME-D-12-00380.1

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Taiwan Society Of Emergency Medicine. (2018). Entrustable professional activity project [online]. Available at: https://www.sem.org.tw/ (Accessed October.22, 2021).

Google Scholar

38. Law, B, Meijers, F, and Wijers, G. New perspectives on career and identity in the contemporary world. Br J Guid Couns. (2002) 30:431–49. doi: 10.1080/0306988021000025637

CrossRef Full Text | Google Scholar

39. Monrouxe, LV, Bullock, A, Tseng, H-M, and Wells, SE. Association of professional identity, gender, team understanding, anxiety and workplace learning alignment with burnout in junior doctors: a longitudinal cohort study. BMJ Open. (2017) 7:e017942. doi: 10.1136/bmjopen-2017-017942

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Armitage, P, Berry, G, and Matthews, JNS. Statistical Methods in Medical Research John Wiley & Sons (2008).

Google Scholar

41. Cicchetti, D. (2017). Choice of Agreement Statistics: A Discussion of the Underlying Biostatistics, with Heuristic Examples from the Vineland-3. Ncs Pearson, Inc, 1–9.

Google Scholar

42. Crossley, J, Johnson, G, Booth, J, and Wade, W. Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ. (2011) 45:560–9. doi: 10.1111/j.1365-2923.2010.03913.x

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Crossley, J, and Jolly, B. Making sense of work-based assessment: ask the right questions, in the right way, about the right things, of the right people. Med Educ. (2012) 46:28–37. doi: 10.1111/j.1365-2923.2011.04166.x

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Holmboe, ES. Realizing the promise of competency-based medical education. Acad Med. (2015) 90:411–3. doi: 10.1097/ACM.0000000000000515

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Kang, AJ, Gielissen, K, and Windish, D. Using an entrustable professional activity to assess consultation requests called on an internal medicine teaching service. MedEdportal. (2019) 15:10854. doi: 10.15766/mep_2374-8265

CrossRef Full Text | Google Scholar

46. Tai, J, Ajjawi, R, Boud, D, Dawson, P, and Panadero, E. Developing evaluative judgement: enabling students to make decisions about the quality of work. High Educ. (2018) 76:467–81. doi: 10.1007/s10734-017-0220-3

CrossRef Full Text | Google Scholar

47. Tai, JH-M, Canny, BJ, Haines, TP, and Molloy, EK. The role of peer-assisted learning in building evaluative judgement: opportunities in clinical medical education. Adv Health Sci Educ. (2016) 21:659–76. doi: 10.1007/s10459-015-9659-0

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Bearman, M. Prefigurement, identities and agency: The disciplinary nature of evaluative judgement In: Developing Evaluative Judgement in Higher Education : Routledge (2018).

Google Scholar

49. Shakeri, K, Fallahi-Khoshknab, M, Khankeh, H, Hosseini, M, and Heidari, M. Knowledge, attitude, and clinical skill of emergency medical technicians from Tehran emergency center in trauma exposure. Int J Crit Illn Inj Sci. (2018) 8:188. doi: 10.4103/IJCIIS.IJCIIS_33_18

CrossRef Full Text | Google Scholar

50. Beeson, MS, Warrington, S, Bradford-Saffles, A, and Hart, D. Entrustable professional activities: making sense of the emergency medicine milestones. J Emerg Med. (2014) 47:441–52. doi: 10.1016/j.jemermed.2014.06.014

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Jarrett, JB, Berenbrok, LA, Goliak, KL, Meyer, SM, and Shaughnessy, AF. Entrustable professional activities as a novel framework for pharmacy education. Am J Pharm Educ. (2018) 82:6256. doi: 10.5688/ajpe6256

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Bentley, SV, Peters, K, Haslam, SA, and Greenaway, KH. Construction at work: multiple identities scaffold professional identity development in academia. Front Psychol. (2019) 10:628. doi: 10.3389/fpsyg.2019.00628

PubMed Abstract | CrossRef Full Text | Google Scholar