A novel taxonomy for facilitating in-depth comparison of continuing engineering education practices

Introduction: This study addresses the urgent need for standardized frameworks in Continuing Engineering Education (CEE) to support lifelong learning in the rapidly evolving global workforce. Significant events, such as the Digital Transformation and the rise of artificial intelligence, have highlighted the demand for adaptable and diverse learning systems, especially in engineering education. On the one hand, the World Economic Forum's, 2023 Future of Jobs Report predicts a transformation in 44% of workers’ core skills within 5 years, with engineering facing substantial labor shortages. On the other hand, the Paris Agreement’s call for sustainable development necessitates a workforce with skills aligned with the green transition.

Methods: The research introduces an innovative framework taxonomy that categorizes and organizes CEE programs by integrating standardized terminologies. It focuses on critical elements such as resourcing, organizational models, and program development strategies to provide a comprehensive structure that supports consistency and comparability across diverse educational contexts.

Results: Unlike existing models, the proposed framework enhances cross-institutional learning and dissemination of best practices. It identifies key components required for effective CEE management, filling a critical gap in the literature. The study’s output is a taxonomy for discussing and comparing CEE institutional approaches and practices to advance the field and foster a global CEE community committed to excellence in engineering education.

Discussions: This study equips educators, policymakers, and industry leaders with a practical tool to design, implement, and scale CEE programs. It ultimately fosters a skilled workforce prepared to meet the challenges of future technological and sustainable transitions, supporting the development of a standardized approach to CEE.

1 Introduction

Despite the growing importance of Continuing Engineering Education (CEE) in addressing skills gaps driven by technological advancements and sustainability demands, there is a lack of standardized frameworks and taxonomies to categorize and manage CEE programs effectively categorize and manage CEE programs. This absence hinders cross-institutional collaboration, knowledge sharing, and the ability to assess the quality and effectiveness of CEE initiatives. Existing frameworks lack a common language, complicating CEE activities’ design, delivery, and evaluation and the design, delivery, and evaluation of CEE activities across educational contexts. Global events, such as the rapid adoption of artificial intelligence tools, widespread uptake in digital education, and a dynamic labor market (in terms of the nature of jobs, roles, and occupations) across diverse workforces, necessitate an innovative response to upskill and reskill those in the workplace. This urgency is apparent, with 44% of workers’ core skills expected to adapt by 2030. Moreover, engineering is a crucial occupation with identified labor shortages (World Economic Forum, 2023). Additionally, 18 million more green transition jobs are expected to be created worldwide in our effort to achieve the Paris Agreement (International Labour Organization, 2018). These developments highlight the need to boost the Continuing Education and Lifelong Learning systems and adapt them to be more diverse and agile. For the purpose of this study, Continuing Education (CE) is defined as the “professional development activities in which an employee, typically with a completed academic degree, seeks further learning from a higher education institution” (Friedman and Phillips, 2004). Lifelong learning (LLL) refers to the learning activities aimed at improving knowledge, skills, and competencies throughout life, having an impact on both professional and personal development (Froehle et al., 2022).

Skills and competencies are critical competitive factors in the industry. As companies continually adapt to rapid technological advancements that consistently impose new requirements on engineers, the significance of CEE cannot be overstated. Industries are transforming in response to these technological shifts and changing market demands, making it increasingly critical for professionals to update their skills and knowledge. Despite the acknowledged value of CEE, there remains a notable lack of standardization in how continuing education activities are categorized and delivered (Thwe and Kálmán, 2024). This absence of a unified framework complicates the ability of individuals and organizations to assess the quality, relevance, and effectiveness of CEE learning activities and challenges providers of CEE to design, develop, and compare activities, such as through credit systems like European Credit Transfer and Accumulation System (ECTS), levels of progression, and further also a common language for researchers to study CEE learning activities. Although this diversity provides opportunities for cross-fertilization of ideas and creativity, it can also result in fragmentation of the field and duplication of effort. Therefore, it becomes essential to develop a taxonomy that creates a language with common terminology to facilitate more precise communication and understanding among stakeholders involved in CEE, enabling more effective responses to these evolving demands.

Research within CEE is wide-ranging, diverse, and interdisciplinary; it builds on exciting and innovative research and practice being conducted in a broad range of educational and professional engineering settings across the globe. The timely and efficient capacity building of professionals and employees is becoming a priority in many countries, not only to ensure that no one is left behind in this era of great transformation but also that the labor market has the skills needed to meet the needs of this new economic, environmental, and social setup. Although influenced by unique country-specific contexts, governments and educational institutions actively seek appropriate responses to these challenges, often in isolation and without a clear guide to designing their CEE and LLL systems best.

The diversity of CEE activities, from formal to informal to non-formal learning activities, along with various delivery methods, such as online platforms, workshops, seminars, courses, and work-based learning, presents a complex ecosystem. This complexity also challenges participants in effectively navigating their learning paths and providers in efficiently designing and marketing their offerings. Furthermore, employers and accrediting bodies struggle to recognize and value CEE achievements due to lacking this common language and standards. These challenges suggest the need for a taxonomy of continuing engineering education activities and their delivery methods. A well-defined taxonomy would provide a standardized language that categorizes CE activities and delivery methods, facilitating stakeholder communication. However, no taxonomy, framework, or standardized language exists for comparing and managing CEE learning programs and interventions. A taxonomy is a hierarchical scheme that classifies terms within a field using a standardized language (Finelli et al., 2015). In the context of this research, a framework is taken to be a system of classification that can categorize and organize how institutions and program teams manage and deliver CEE (Nickerson et al., 2013). Additionally, there are limited conceptualizations around frameworks and taxonomies in Continuing Education (Jarvis, 1996, 2004) and recognition of the complexities in developing such taxonomies, as well as being focused on the learning [micro-level] activities rather than institutional decision-making (Lindsay and Richard, 1972). This paper addresses this gap by introducing a taxonomy around CEE models, and focusing on effectively managing these programs to encourage the exchange of best practices.

Recent studies underscore the need for a taxonomy in continuing education. Initial works in 2023 examined meso-level factors within academic institutions, primarily in Europe (Caratozzolo et al., 2023; Gomez Puente et al., 2023). These studies develop a systematic understanding of continuing education by categorizing key aspects into drivers and opportunities, organizational arrangements, types of offerings, and accessibility. These classifications not only refine the approach to continuing education but also provide a foundation for effectively comparing different educational systems and practices. Thus, a well-defined taxonomy is essential for categorizing and simplifying complex information and fostering dialogue and collaboration across educational landscapes. It acts as a critical tool in advancing the field of continuing education by enabling a clearer understanding of existing patterns and emerging trends and providing a structured framework that supports ongoing research and development in this vital area.

This study builds upon a 2023 comparative analysis of CEE at diverse universities in different countries, emphasizing the necessity for a standardized language to facilitate cross-country and cross-institutional knowledge sharing (Gomez Puente et al., 2023). This paper aims to unveil a new framework that could be adopted across different countries and continents, highlighting categories such as resourcing (of learning), organizational/business models, and program development strategies. The impact of this framework lies in facilitating the exchange of knowledge and best practices among actors in CEE and allowing them to embark much more quickly on collaborative projects in this space. Furthermore, the fact that CEE is on top of the agendas for many educational institutions and universities, it becomes essential to have a framework that allows for self-assessment of the organizations, for mapping and comparing CEE strategies, and, consequently, for the purpose of benchmarking. Finally, this taxonomy helps bridge the gap between academia and the labor market as education can be better aligned with industry needs at employers’ services.

Research Question (RQ): What are the critical components of a framework for comparing Continuing Engineering Education (CEE) programs?

2 Overview

2.1 The complexity of discussing continuing engineering education (CEE) practices

CEE offerings support ongoing learning, upskilling, and reskilling development to keep engineers abreast of evolving technologies, methodologies, and industry standards (Pérez-Foguet and Lazzarini, 2019; Kimmel et al., 2022; Li, 2022; Leon, 2023). These programs may include workshops, seminars, courses, certifications, conferences, and other professional development activities tailored to meet the specialized needs of engineers across various disciplines and career stages. However, discussing and comparing these practices presents inherent complexities due to the multifaceted nature of engineering education, which poses challenges when discussing and comparing CEE practices. Engineering encompasses diverse fields with specialized knowledge areas and skill sets (Caratozzolo et al., 2022; Skills Future Singapore, 2022). Additionally, CEE involves various stakeholders, including educational institutions, industry partners, professional associations, and regulatory bodies, each with distinct perspectives, goals, and priorities. Moreover, the rapid pace of technological advancements and industry changes further complicates the landscape, requiring CEE programs to adapt continuously to emerging trends and developments.

Traditional approaches to evaluating CEE programs often lack standardized language and taxonomies, which hinders effective communication and comparison of practices across institutions and countries (Lawanto et al., 2017). Without clear definitions and categorizations, assessing, comparing, and disseminating best practices in CEE becomes challenging. Standardized language ensures consistency in terminology, while taxonomies provide structured frameworks for organizing and classifying CEE practices based on critical criteria such as program objectives, delivery methods, learner outcomes, and assessment metrics. Standardized vocabularies and taxonomies can facilitate more efficient knowledge sharing, collaboration, and quality assurance in CEE initiatives.

2.2 Motivation for standardized language

In the context of CEE within the Industry 4.0 and Education 4.0 frameworks, standardized language facilitates effective communication and collaboration among professionals across diverse engineering disciplines (Chakraborty et al., 2023). The necessity for standardized language in discussing CEE practices stems from the urgency to adapt engineering education to meet the demands of a rapidly changing global workforce (Lagorio et al., 2023; Thwe and Kálmán, 2024).

The World Economic Forum's, 2023 Future of Jobs report highlights the impending evolution of workers’ core skills, emphasizing the critical role of CEE in addressing labor shortages and facilitating the green transition outlined by the Paris Agreement (Horowitz, 2016; World Economic Forum, 2023). As the skills required in the workforce evolve, CEE programs play a crucial role in upskilling and reskilling engineers to meet these changing demands. Researchers can identify common themes, emerging trends, and best practices in CEE by systematically examining various sources. These insights inform the development of a taxonomy with standardized language and an associated framework by highlighting key concepts, terminology, and areas where consensus or divergence exists among practitioners and researchers. Consequently, standardized language, a taxonomy, and a framework developed through this process are more likely to be comprehensive, inclusive, and reflective of the diverse perspectives and practices within the field of CEE (Baukal, 2022; Kimmel et al., 2022; Kubrushko and Kozlenkova, 2019).

2.3 The CEE taxonomy and its importance

A taxonomy is an essential prerequisite for further research and practice in CEE. The importance of a CEE taxonomy lies in the organization and categorization of diverse elements inherent in these educational initiatives (Finelli et al., 2015). It helps to break down complex phenomena into manageable and coherent structures, making it easier for stakeholders to understand and navigate the landscape of CEE programs. Moreover, a taxonomy allows for the standardized language development, promoting clarity and consistency in discussions surrounding CEE practices. Through the implementation of a taxonomy, stakeholders in engineering education can navigate the complexities of CEE more effectively, driving innovation and advancement in the field (Coleman and Radulovici, 2020; Caratozzolo et al., 2023). The taxonomy’s structure helps stakeholders identify critical components, relationships, and patterns within CEE programs, ultimately driving innovation and advancement in the field by promoting informed decision-making and strategic planning.

3 Methodology

The nature of the research question is predominantly qualitative, with the first objective of determining terms and concepts that describe CEE programs. When viewed in terms of any taxonomy of terms enabling the enhanced exchange of ideas, then the components need to encompass both (a) the educational aspects (what, who, where, and how it is delivered) and (b) include contextual components that speak to the way it is delivered. Specifically, the methodology is sequential mixed methods, with a qualitative, inductive analysis of existing case studies to generate the initial taxonomy and a quantitative step to count occurrences of identified terms within the literature to examine whether alternative wording in this initial taxonomy was supported through frequency of use of terms found in existing literature; the details of this are presented below.

The methodology for this research reflects that of Finnelli et al. used to develop a taxonomy for Engineering Education Research (EER) (Finelli et al., 2015). The EER taxonomy was developed to support a range of groups (researchers, novices in the field, journal editors, and funders) in having a standardized terminology. The development of the EER taxonomy was guided by two main principles: first, that it would be helpful, and second, the use of an inclusive process. Therefore, in line with (Hedden, 2010), a series of subject matter experts from different countries were engaged, along with studying the existing corpus in Engineering Education. In particular, Finelli (ibid) developed an initial draft version refined through conference events and engagement with groups of experts.

For this taxonomy around CEE, the authors sought to produce a helpful taxonomy through an inclusive process involving representatives from eight countries. The utility of this is that it is relevant to novice researchers, helps to “map the field” and will allow researchers and practitioners “to situate their individual research initiatives in the broader field, see connections and synthesize ideas, … and plan future work” (Finelli et al., 2015, p. 366). Specifically, the methods for the Research Question were: (i) to inductively re-analyze an existing comparative case-study data set, version 1.0 (Gomez Puente et al., 2023) to determine a broader range of concepts outlined and the language used, (ii) these terms were then inductively synthesized into an updated taxonomic framework, version 2.0, (iii) a mapping review was undertaken using originally-generated terms along with synonyms to determine the frequency of use of those terms, (iv) a subsequent updating of a taxonomic framework based on a review of frequency, version 2.1. Such a process aligns with the analysis of the existing corpus and the engagement of subject matter experts (Hedden, 2010; Finelli et al., 2015).

For the first step, eight previous case studies were re-analyzed, and a collaborative, online workshop was held between the eight co-researchers to list additional terms relevant to CEE. This workshop expanded the set of terms from 21 to 92. Subsequently, one researcher inductively grouped these additional and pre-existing terms, moving from the original five categories of Gomez Puente et al. (2023) to four top-level categories: (1) Drivers and Opportunities; Organizational Arrangements; (2) Types of CEE offering; and (3) Accessibility of CEE; this grouping synthesized previous top-level categories and provided a broader mapping of the CEE field. The other research team members evaluated and validated this grouping through a collaborative online review. In particular, the updated taxonomy was shared with co-authors 7 days before the online review. In the online review, the other co-researchers reviewed the overall taxonomy, specifically structure and language, and discussed whether the categories were (a) representative of the case studies and their institutional practices, (b) provided an appropriate clustering of the terms that had been generated from the previous workshop in which they had been involved; and (c) whether the current categorization would be helpful to [linking back to Finelli et al.’s (2015) principle]. This review saw broad consensus through confirmational discussion of each category. However, the review did lead to minor changes, such as the addition of ‘blended’ to ‘Delivery modes, and the change of the fourth category from ‘Accessibility of CEE’ to ‘Accessibility and inclusion of CEE’ that saw an expansion of terms in this category to reflect ‘Assistive technology’ and sub-terms, ‘Economic support’ and sub-terms. And ‘Geographical support’ and sub-terms.

A broad and exploratory mapping was performed to review the literature related to critical topics. This mapping aimed to describe the current state of knowledge in these areas. The search strategy focused on obtaining documents from the Scopus database, focusing on English language publications from 2015 onwards (Lockwood et al., 2019; Mak and Thomas, 2022). Each retrieved document was carefully examined to assess its relevance to the study objectives.

This process of retrieving and assessing these documents was performed by prioritizing documents published in English to ensure consistency across the international research team. In addition, only works published after 2015 were included to maintain relevance and reflect current trends in CEE. Key search terms included “taxonomy,” “lexicon,” “framework,” “labor education,” “continuing engineering education,” “continuing education,” “professional development” and “lifelong learning,” along with their synonyms. Only papers indexed in Scopus were considered, ensuring academic rigor in the selection process. Additionally, documents that significantly deviated from key topics or did not address relevant terms were excluded from consideration. Research published before 2015 was also excluded to focus on contemporary scholarship and those not in English to maintain consistency across the board. These criteria helped streamline the selection process, ensuring that only the most relevant and highest-quality papers were reviewed to inform the development of a comprehensive and up-to-date taxonomy.

Subsequently, a comprehensive base taxonomy of terms was compiled based on the authors’ previous work and studies. This initial taxonomy served as a basis for identifying synonyms and similar terms using Large Language Model LLM-based methods (Thießen et al., 2023). This aims to broaden the range of search terms and ensure the inclusion of the most prominent and widely recognized terms within the defined taxonomy.

After compiling synonyms and analogous terms related to the base taxonomy, a comprehensive review of the collected literature was conducted to identify works containing keywords related to the base taxonomy. The aim was to give priority to terms that were considered popular and current within the field. The objective was to integrate these terms into the taxonomic structure, refining it to reflect the contemporary standardized language of the study area. This iterative process involved modifying the base taxonomy to incorporate terms better aligned with prevailing terminology and thematic relevance, ensuring the resulting taxonomy was accessible and up-to-date. Once the collected works’ content, themes, and subthemes were identified, a detailed analysis was conducted to validate and refine the underlying taxonomic structure. The rigorous validation process involved comparing the identified terms with a complete corpus of related literature, focusing on integrating the most relevant and prevalent terms to enrich the taxonomic framework. The decision-making around this reflected the following decision criteria:

1. An alternative term, highlighted by frequency analysis of literature, would be adopted if it still reflected current and emergent terminology;

2. More general terms were kept, rather than terms or phrasing that was potentially limiting (e.g., workshop as just one method to develop a CEE educator, so while this term is prevalent in literature, then it was not adopted);

3. Terms reflecting current and emergent usage were preferred, such as up- and re-skilling (over professional development);

4. Redundant terms of the hierarchy were removed, such as having ‘Masters award’ in the category of ‘University full-awards’;

5. Consideration was given to terms, based on the range of nationalities of co-researchers, to find terms that would be as international as possible and not tied to one nation or institution;

6. That the most frequent term had not sufficiently captured the original term’s essence.

Finally, the additional components from this literature search were used to refine the taxonomy, version 2.1, presented in Section 4.3.

A methodological limitation of this research is that it is mainly European-focused on individuals within academic-only institutions. This emergent taxonomy needs further validation within the co-researchers institutions. Additionally, the intention is to use future conferences to share, discuss, and further refine the taxonomy with a broader range of participants (non-academic institutions, different countries, and continents) (Finelli et al., 2015).

4 Findings and discussion

4.1 Initial taxonomy refinement

An early form of the categorization was presented as part of the 2023 findings of Gomez Puente et al. which focused just on meso-level factors (so a subset of data and smaller in scope than the current RQ) (Gomez Puente et al., 2023). These findings were subsequently presented in a taxonomic format. The analysis of the comparative cases from eight (8) academic institutions with the broader RQ used in this research led to terms and categorization into four: (a) Drivers and opportunities, (b) Organizational arrangements, (c) types of CEE offering, and (d) Accessibility of CEE. Table 1 shows the uppermost three levels for each area for ease of display. The complete taxonomy has a maximum of six (6) levels. This refinement occurred when the researchers compared the CEE units of their respective universities and found many differences, mainly in how CEE is organized. In some countries, a clear separation of free basic education on the one hand and fee-based continuing education on the other hand needs to be observed which entails different organizational and financial settings. In case of such a separation, the university’s teaching staff must not be involved in continuing education as part of their teaching obligation but has to do it as a paid side job. Consequently, it may be appropriate to outsource the CEE activities to a private but associated company, which again leads to further differentiations. This and other aspects finally led to a deeper taxonomy tree with six levels that apply to universities as CEE providers.

Table 1

Table 1. Emergent taxonomy from a broader analysis (version 2.0).

4.2 Findings in the clustering process for the mapping review

This subsection presents the findings derived from the clustering process undertaken during the mapping review. This clustering process was integral to the research as it aimed to group related documents based on thematic similarities. It offered a structured way to analyze patterns in the literature on Continuing Engineering Education (CEE) and lifelong learning. By clustering documents around the key terms: “taxonomy,” “lexicon,” “framework,” “workforce education,” “continuing engineering education,” “continuing education,” “professional development,” and “lifelong learning” the researchers could discern relationships between different studies and identify recurring themes. The analysis encompassed a total of 83 documents distributed across various thematic categories: taxonomy (8), lexicon (2), framework (7), workforce education (13), continuing engineering education (24), continuing education (14), professional development (7), and lifelong learning (8). These documents formed the basis for identifying patterns, trends, and insights pertinent to the overarching objectives of the research endeavor, and the clustering process helped validate and refine the taxonomy by aligning the most relevant concepts from the literature with the research objectives. The clusters provided a clearer understanding of how CEE is approached across various institutions and frameworks, contributing to developing a more standardized language within the field. Ultimately, the clustering analysis allowed for a more focused exploration of the literature, ensuring that the taxonomy was both comprehensive and reflective of current trends.

To find a correlation between the themes and subthemes of the works found in the mapping, all this data was grouped based on the work of Azofeifa et al., which seeks to glimpse in a defined space how related or similar these works are from a general point of view of the whole (Azofeifa et al., 2022).

Using the themes of taxonomy (T), lexicon (L), framework (F), workforce education (WE), Continuing engineering education (CEE), continuing education (CE), professional development (PD), and lifelong learning (LL), in addition to classifying the works by topic, an extra classification was made where the other topics from the list are taken and if they are presented in the works they are taken as subtopics, taking into account that each work can use one or more of the options it has available among the subtopics. To compare these categorizations effectively, we devised a metric that assigns each item a value to each work based on the presence of the main topic (assigned a value of 2), subtopics (assigned a value of 1), or neither (assigned a value of 0). This value system provides a quantitative measure of thematic relevance within the dataset, facilitating subsequent analyses.

The computed distances between documents enable us to perform cluster analysis in a multidimensional space using the L2 Euclidean norm (Rump, 2023). We determined the optimal number of clusters using the elbow method, identifying the optimal cluster number of three (k = 3). Then, we applied the k-means algorithm to group the data effectively. This algorithm partitioned the works into distinct clusters based on their proximity in the multidimensional space, ensuring a balance between cluster compactness and member distribution. Initially represented by a dendrogram, the resulting clusters were further visualized using t-SNE (t-distributed stochastic neighbor embedding) to project the multidimensional data onto a 2D plane (Figure 1). This t-SNE method retains local relationships from higher dimensions in the lower-dimensional space, revealing intricate relationships and groupings among the works based on their thematic affiliations. Supplementary Data Sheet 1 contains the full mapping references represented by the “identification codes” used in Figure 1.

Figure 1

Figure 1. The t-SNE method, with the three distinct clusters identified by the dendrogram and their correlation distribution.

Identifying three distinct clusters was not merely a technical outcome but a significant insight into the field of CEE and lifelong learning. Each cluster represented cohesive groups of studies that shared common themes, offering a more nuanced understanding of the different approaches to professional development and education. For instance, one cluster might emphasize the role of frameworks in guiding lifelong learning, while another could focus on specific educational models in engineering. These clusters provide valuable new insights by highlighting gaps in the literature or emerging trends that were previously overlooked.

Through this visualization, we observed three distinct clusters, each representing cohesive groups of works sharing common themes and subthemes. For example, the work of lifelong learning LL1 and lexicon L1, or the works of continuing engineering education CEE5, CEE6, and CEE7, among others, reveal close relationships. These connections could potentially unveil areas of opportunity for new research, sparking curiosity and anticipation for future developments. As a result, the clustering process contributes to the refinement of the taxonomy, helping to organize the diverse elements of CEE and fostering more effective communication and collaboration across educational institutions, addressing the broader research questions outlined in Section 4.1.

To validate and refine the foundational taxonomic structure in alignment with terms identified in the literature, we initially sought synonyms or analogous terms to those within the base taxonomy. Employing methods rooted in Large Language Models (LLM), we expanded the scope of search terms, aiming to include the most pertinent and widely recognized terms within the definition of the base taxonomy (Thießen et al., 2023). This process enriched the search terms pool and enhanced subsequent literature searches” comprehensiveness.

Following compiling a list comprising synonyms and akin terms to those present in the base taxonomic structure, we conducted searches within the previously identified corpus of related literature. This involved querying for the identified synonyms, similar terms, and base terms within the list of associated works, subsequently tallying and identifying those works containing these terms. After this step, we meticulously analyzed the outcomes of these searches to refine the foundational taxonomic structure. Emphasis was placed on integrating terms that best aligned with the lexicon of the field, considering factors such as prevalence, relevance, and contemporaneity. This iterative process ensured that the resultant taxonomic framework remained both accessible and current, reflecting the evolving terminology and conceptual landscape of the field.

4.3 Synthesized taxonomy

As outlined in the methodology section above, the frequency analysis of terms in existing literature and findings from the clustering process played a critical role in refining the taxonomy, directly influencing the transition from version 1.0 to version 2.0. By organizing the literature into distinct clusters based on shared themes, the research team could identify patterns that were not apparent through traditional qualitative analysis alone. These clusters revealed recurring concepts and terminologies across studies, highlighting gaps and redundancies in the initial taxonomy. For example, the clustering process identified frequent use of terms related to “upskilling” and “reskilling,” prompting the inclusion of these terms in the updated taxonomy version. Similarly, by recognizing overlaps in the literature on delivery methods, such as “blended learning” and “hybrid learning,” the researchers were able to simplify and clarify the categorization of these concepts within the taxonomy. This iterative process ensured that the taxonomy not only reflected the most current trends in CEE but also provided a more structured and comprehensive framework for categorizing educational practices. Based on these discussions, the following changes were made at the top level (Table 2). Additionally, some terms were adapted based on discussions. Supplementary Data Sheet 2 contains information on the more frequent alternative terms and the corresponding references included in Table 2.

Table 2

Table 2. Summary of changes made to the taxonomy.

In the review discussions, some terms were still felt that there was insufficient consensus amongst the research, such as ‘3.a. Naming’, or were terms that would need an accompanying definition to ensure consistency in use and understanding, such as ‘3.a.i. Course’. Moreover, while ‘3.b.ii.1. non-formal’ and ‘3.b.ii.2 informal’ are recognized forms of lifelong learning, so they apply to CEE; there was not sufficient clarity amongst researchers as to how these terms applied within CEE, so definitions are required, along with further engagement with the CEE community to determine how these fit into CEE (Johri, 2022).

There were some terms, such as ‘3.b.1.c. Sub-degree awards’ were included to reflect practices in some countries and may not be universally applicable. For some, CEE is seen as post-Bachelors education, so the use of ‘3.a.v Degree qualification’ indicating Masters would be the main focus. However, in some countries, a student can iteratively progress towards a bachelor’s degree through sequential studies and awards, with potential study breaks in between (Lester, 2015; Bohlinger, 2019):

• VET/Further Education—HNC [Higher National Certificate]—EQF4.

• VET/Further Education—HND [Higher National Diploma]—EQF5.

• University - Bachelors (ordinary).

• University - Bachelors (Honors)—EQF 6.

In this context, further refinement of the taxonomy is required to ensure it can broadly capture practices from different national systems while recognizing that an international taxonomy may not reflect all national and institutional considerations.

Figure 2 reflects a refinement of Table 1, with the updated version 2.1 taxonomy for the uppermost three levels and maintaining the four key categories: Drivers and opportunities, Organizational arrangements, Types of offerings, and Accessibility. The full taxonomic organization (six levels) is in Supplementary Data Sheet 3. However, the additional three levels in this full taxonomy only apply to the university as a provider (2.a.i). At the next level, the first aspect is where CEE is placed in the university organization: It can be a central unit or organized in a distributed way by faculties or departments, or it may be placed in an associated private company. The second aspect addresses the educators of CEE: They may be regular teaching staff who teach in CEE as part of their obligation, or they may teach as a paid side job. If external educators do teaching, it can be organized individually or in partnership with an external institution. Thirdly, teaching seasoned professionals may require additional training for educators, which can be part of regular academic preparation or the use of external experts. Finally, the courses may be aligned to the university calendar or independent.

Figure 2

Figure 2. Updated taxonomy after literature review, uppermost three levels (version 2.1), and four key categories.

5 Discussion

This study’s output is a taxonomy for discussing and comparing CEE institutional approaches and practices to advance the field and foster a global CEE community committed to excellence in engineering education. This ‘taxonomy enables discussion and comparison of CEE institutional approaches and practices to advance the field and foster a global CEE community committed to excellence in engineering education. The results of this study provide guidelines for both institutions and policymakers to critically analyze CEE practices in their own higher education institutions. Moreover, it provides a taxonomy in levels to facilitate the revision process in more detail and stimulate collaboration and knowledge exchange.

Despite the promising preliminary insights from this research study, there are some limitations to point out. Firstly, the research is framed in earlier work on analyzing CEE practices at the meso-level with a focus on university-wide practices. Furthermore, the analysis of the comparative case studies is based on researchers from eight academic institutions. That may not necessarily represent either the institutions’ views or the developments at the national level. Also, although these cases provide more in-depth observations, they are not broadly validated with similar higher education institutions elsewhere. Therefore, results may be considered limited to the context of this research. In this regard, it is worth noting that the taxonomy collects the practices of these institutions but may not reflect the national or international CEE practices. We suggest that the standardized terms and framework introduced in this study can be gradually integrated into both undergraduate and graduate engineering curricula by embedding them in existing course structures related to professional development, engineering management, and lifelong learning.

Secondly, the proposed taxonomy focuses on categories that serve to develop a structure that may suit the purpose of the institutions participating in this study. However, despite the multidimensional approach to include relevant categories such as business models, resourcing, and development strategies, the taxonomy does not serve all analysis purposes, failing to provide a broader scope to explain consequences for other levels, such as courses. Importantly, as detailed in section 4.3, some terms (e.g., course) carry different meanings in different countries, so definitions for each term will be required to reap the full benefits of this taxonomy.

Also, limitations regarding the methodology applied in this study need to be addressed. In this regard, the methodology chosen has taken both a qualitative and quantitative dimension. Qualitative methods were meant to revise terms and concepts to define elements of CEE programs. Also, well-defined methodologies to develop taxonomies, such as Finelli’s, were used, and a non-systematic literature review was conducted to help re-analyze a previous comparative study to determine a broader range of concepts to define the taxonomy used in this study (Finelli et al., 2015). The quantitative dimension made it possible to fine-tune the framework based on a survey of original CEE terms and synonym frequencies. Despite the thoroughness of the approach, the proposed taxonomy would gain more value when compared with the elements accepted by the international community and the body of knowledge in this field.

Together, the above limitations suggest that further refinement of the taxonomy will require a broader range of international collaboration among CEE stakeholders (industry, professional/accreditation bodies, CEE learners) and accrediting bodies and more representation from formal education (a more comprehensive range of universities and inclusion of VET institutions). Consequently, the subsequent phases of this research will involve a broader, more international engagement to refine the taxonomic framework further. Moreover, it will require the creation of definitions to accompany those terms.

6 Conclusion

The need for a standardized language to allow practical discussions and collective progress in Continuing Engineering Education has created a novel taxonomy with associated definitions. Furthermore, the methodology followed to propose a taxonomy is promising, as combining qualitative and quantitative methods accompanied by analyzing universities’ cases allows for the first verification of findings. To the authors’ knowledge, this is the first systematic organization of terms relevant to CEE, so it represents a significant contribution to the contemporary conceptualization of CEE.

Practically, this taxonomy at a macro- (broader national and international perspectives), meso- (institutional decision-making), and micro-level will support more efficient sharing of practices and meet the demand for engineers to reskill and upskill to meet the current and ongoing learning needs. At the national policy level, the taxonomy will allow for greater ease in consistently mapping practices, where good practices exist, where to target support, and where to invest in addressing barriers. Specifically, this mapping will serve the purpose of benchmarking among international institutions, which also serves quality assurance goals relevant to all universities. Similarly, at the institutional level, the taxonomy will allow individual organizations to assess their CEE policy and strategy, thereby identifying strengths as well as areas for enhancement. At a department or individual level, the taxonomy will allow for greater alignment of planned or emergent activity against drivers and options (lower levels of hierarchy) to influence effective decision-making around CEE offerings.

Importantly, this taxonomy and its standardized terms will allow for consistent modeling of the CEE ecosystems and determine the interplay between these factors and which are the most influential. Future research to map and visualize the CEE systems will be an important step in modeling how changes in taxonomy (factors) influence success and outcomes from different institutional approaches to CEE.

In conclusion, this paper is the first relevant step towards refining terms for shared understanding in categories and classification. The taxonomy can be a helpful tool for more consistently and effectively benchmarking institutions on current CEE practices. Finally, the taxonomy that covers macro-, meso- and micro-factors enables organizations and their partners to construct training and similar re-and upskilling professional development for CEE programs by aligning education levels with the needs of the job profile.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the [patients/participants OR patients/participants legal guardian/next of kin] was not required to participate in this study in accordance with the national legislation and the institutional requirements.

Author contributions

PC: Conceptualization, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing. CS: Conceptualization, Formal analysis, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing. SG: Conceptualization, Investigation, Resources, Validation, Writing – original draft, Writing – review & editing. MM: Conceptualization, Investigation, Validation, Visualization, Writing – original draft, Writing – review & editing. BN: Conceptualization, Investigation, Resources, Validation, Writing – original draft, Writing – review & editing. H-UH: Conceptualization, Investigation, Resources, Validation, Writing – original draft, Writing – review & editing. JA: Conceptualization, Data curation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This research was partly funded by the Writing Lab and the Challenge-Based Research Funding Program, Grant no. I030-IFE002-C2-T1-E, both of the Institute for the Future of Education, Tecnologico de Monterrey, Mexico.

Acknowledgments

The authors would like to acknowledge the technical support of Sofia Margarita Hernandez-Munoz, Abiel Borja, Fernando Arana-Salas (student internships), and Fernando Guzman (project engineer) for their caring support in the data curation. They would also like to acknowledge the financial support of Writing Lab and the Challenge-Based Research Funding Program, Grant no. I030-IFE002-C2-T1-E, both of the Institute for the Future of Education, Tecnologico de Monterrey, Mexico, in producing this work.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer JM-H declared a shared affiliation with the author(s) PC, JA to the handling editor at the time of review.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2024.1444595/full#supplementary-material

References

Azofeifa, J. D., Noguez, J., Ruiz, S., Molina-Espinosa, J. M., Magana, A. J., and Benes, B. (2022). Systematic review of multimodal human–computer interaction. Informatics 9:13. doi: 10.3390/informatics9010013

Crossref Full Text | Google Scholar

Baukal, C. E. Jr. (2022). Continuing engineering education handbook : Continuing Engineering Education Handbook, 366.

Google Scholar

Bohlinger, S. (2019). Ten years after: the “success story” of the European qualifications framework. J. Educ. Work. 32, 393–406. doi: 10.1080/13639080.2019.1646413

Crossref Full Text | Google Scholar

Caratozzolo, P., Smith, C. J. M., Munoz-Escalona, P., and Membrillo-Hernández, J. (2022). ‘Exploring engineering skills development through a comparison of institutional practices in Mexico and Scotland’, in SEFI 2022 - 50th annual conference of the European Society for Engineering Education. Barcelona, Spain, pp. 1878–1883

Google Scholar

Caratozzolo, P., Azofeifa, J. D., Mejía-Manzano, L. A., Rueda-Castro, V., Noguez, J., Magana, A. J., et al. (2023). ‘A matrix taxonomy of knowledge, skills, and abilities (KSA) shaping 2030 labor market’, in 2023 IEEE Frontiers in education conference (FIE). (College Station, TX, USA), 1–8.

Google Scholar

Chakraborty, S., Gonzalez-Triana, Y., Mendoza, J., and Galatro, D. (2023). Insights on mapping industry 4.0 and education 4.0. Front. Educ. 8:1150190. doi: 10.3389/feduc.2023.1150190

Crossref Full Text | Google Scholar

Coleman, C. O., and Radulovici, A. E. (2020). Challenges for the future of taxonomy: talents, databases and knowledge growth. Megataxa 1, 28–34. doi: 10.11646/megataxa.1.1.5

Crossref Full Text | Google Scholar

Finelli, C. J., Borrego, M., and Rasoulifar, G. (2015). Development of a taxonomy of keywords for engineering education research. IEEE Trans. Educ. 104, 365–387. doi: 10.1002/jee.20101

Crossref Full Text | Google Scholar

Friedman, A., and Phillips, M. (2004). Continuing professional development: developing a vision. J. Educ. Work. 17, 361–376. doi: 10.1080/1363908042000267432

Crossref Full Text | Google Scholar

Froehle, K., Dickman, L., Phillips, A. R., Murzi, H., and Paretti, M. (2022). Understanding lifelong learning and skills development: lessons learned from practicing civil engineers. J. Civil Engin. Educ. 148:04022007. doi: 10.1061/(ASCE)EI.2643-9115.0000068

Crossref Full Text | Google Scholar

Gomez Puente, S. M., Smith, C., Urenda Moris, M., Nørgaard, B., Heiß, H-U., Caratozzolo, P., et al. (2023). ‘Towards building a framework for continuing engineering education in higher education institutions: a comparative study’, in SEFI 2023 - 51st annual conference of the European Society for Engineering Education: Engineering education for sustainability. SEFI 2023 - Dublin, Ireland, pp. 2085–2093

Google Scholar

Hedden, H. (2010). The accidental taxonomist. Medford, USA: Information Today. Inc., May.

Google Scholar

Horowitz, C. A. (2016). Paris Agreement. Int. Leg. Mater. 55, 740–755. Available at: doi: 10.1017/S0020782900004253

Crossref Full Text | Google Scholar

International Labour Organization (2018). World employment and social outlook – trends 2018. Available at: https://www.ilo.org/global/research/global-reports/weso/2018/lang--en/index.htm (Accessed February 6, 2024).

Google Scholar

Jarvis, P. (1996). Continuing education in a late-modern or global society: towards a theoretical framework for comparative analysis. Comp. Educ. 32, 233–244. doi: 10.1080/03050069628867

Crossref Full Text | Google Scholar

Jarvis, P. (2004). Adult education and lifelong learning: Theory and practice. London: Routledge.

Google Scholar

Johri, A. (2022). Lifelong and lifewide learning for the perpetual development of expertise in engineering. Eur. J. Engin. Educ. 47, 70–84. doi: 10.1080/03043797.2021.1944064

Crossref Full Text | Google Scholar

Kimmel, H. S., Rockland, R. H., Carpinelli, J. D., and Spak, G. T. (2022). Effective instructional practices in continuing engineering education. Continuing Engin. Educ. Handbook, 89–127.

Google Scholar

Kubrushko, P., and Kozlenkova, E. (2019). ‘Continuing engineering education: background and development vectors’, In 1st international scientific practical conference ‘the individual and Society in the Modern Geopolitical Environment’ (ISMGE 2019), pp. 397–403

Google Scholar

Lagorio, A., Cimini, C., and Romero, D. (2023). “A reflective framework for understanding workforce evolutionary pathways in industry 5.0” in Advances in production management systems. Production Management Systems for Responsible Manufacturing, service, and logistics futures. eds. E. Alfnes, et al. (Cham: Springer Nature Switzerland).

Google Scholar

Lawanto, O., Uziak, J., Villanueva, I., and Scheaffer, M. (2017). Continuing engineering education: a needs assessment for the introduction of a graduate certificate programme. GJEE, 19, 186–193.

Google Scholar

Leon, R. D. (2023). Employees’ reskilling and upskilling for industry 5.0: selecting the best professional development programmes. Technol. Soc. 75:102393. Available at:. doi: 10.1016/j.techsoc.2023.102393

Crossref Full Text | Google Scholar

Lester, S. (2015). The European Qualifications Framework: a technical critique. Res. Post-Compuls. Educ. 20, 159–172. doi: 10.1080/13596748.2015.1030251

Crossref Full Text | Google Scholar

Li, L. (2022). Reskilling and upskilling the future-ready workforce for industry 4.0 and beyond. Inform. Syst. Front. 24, 1–16. doi: 10.1007/s10796-022-10308-y

PubMed Abstract | Crossref Full Text | Google Scholar

Lindsay, C.A., and Richard, G.H. (1972). Toward a taxonomy for continuing education instructional activities in higher education. Available at: https://eric.ed.gov/?id=ED064587 (Accessed March 21, 2023).

Google Scholar

Lockwood, C., dos Santos, K. B., and Pap, R. (2019). Practical guidance for knowledge synthesis: scoping review methods. Asian Nurs. Res. 13, 287–294. doi: 10.1016/j.anr.2019.11.002

Crossref Full Text | Google Scholar

Mak, S., and Thomas, A. (2022). Steps for conducting a scoping review. J. GraduateMedical Educ. 14, 565–567. doi: 10.4300/JGME-D-22-00621.1

PubMed Abstract | Crossref Full Text | Google Scholar

Nickerson, R. C., Varshney, U., and Muntermann, J. (2013). A method for taxonomy development and its application in information systems. Eur. J. Inf. Syst. 22, 336–359. doi: 10.1057/ejis.2012.26

Crossref Full Text | Google Scholar

Pérez-Foguet, A., and Lazzarini, B. (2019). Continuing professional education in engineering faculties: transversal integration of sustainable human development in basic engineering sciences courses. J. Clean. Prod. 218, 772–781. Available at:. doi: 10.1016/j.jclepro.2019.02.054

Crossref Full Text | Google Scholar

Rump, S. M. (2023). Fast and accurate computation of the Euclidean norm of a vector. Jpn. J. Ind. Appl. Math. 40, 1391–1419. Available at:. doi: 10.1007/s13160-023-00593-8

Crossref Full Text | Google Scholar

Skills Future Singapore (2022). Skills demands for the future economy. Available at: https://www.myskillsfuture.gov.sg/content/portal/en/career-resources/career-resources/education-career-personal-development/Skills_Demand_for_the_Future_Economy_Report_2022.html (Accessed May 4, 2024).

Google Scholar

Thießen, F., D’Souza, J., and Stocker, M. (2023) ‘Probing large language models for scientific synonyms’, In 19th international conference on semantic systems. Leipzig, Germany.

Google Scholar

Thwe, W. P., and Kálmán, A. (2024). Lifelong learning in the educational setting: a systematic literature review. Asia Pac. Educ. Res. 33, 407–417. Available at:. doi: 10.1007/s40299-023-00738-w

Crossref Full Text | Google Scholar

World Economic Forum (2023). The future of jobs report 2023, world economic forum. Available at: https://www.weforum.org/reports/the-future-of-jobs-report-2023/ (Accessed July 28, 2023).

Google Scholar